maria-developers team mailing list archive

["Hollow Man" <hollowman@xxxxxxxxxxxx>]

Hi Vicențiu and Vladislav,


Thanks for all your suggestions, I now have a more comprehensive view of the issues I'm going to face.


I'll start to check other databases, the pandas python library or some other libraries to see if there's any experience that I can learn and fulfill my proposal with tests and situations for corner cases that satisfy points a, b, c, d mentioned and other potential issues.


Songlin
 
 
------------------ Original ------------------
From: &nbsp;"Vladislav Vaintroub"<vvaintroub@xxxxxxxxx&gt;;
Date: &nbsp;Thu, Apr 1, 2021 04:08 PM
To: &nbsp;"Vicențiu Ciorbaru"<vicentiu@xxxxxxxxxxx&gt;; "Hollow Man"<hollowman@xxxxxxxxxxxx&gt;; 
Cc: &nbsp;"maria-developers"<maria-developers@xxxxxxxxxxxxxxxxxxx&gt;; 
Subject: &nbsp;RE: [Maria-developers] GSOC21: MDEV-16375 &amp; MDEV-23143

&nbsp;

 
&nbsp;

&nbsp;

Vicentiu, Hollow Man,

&nbsp;

The order of the values in JSON array is preserved 

https://stackoverflow.com/a/7214312/547065 &nbsp;is a good answer about that, which contains the quote of the specification.

&nbsp;

So that’s not a tricky case

&nbsp;

From: Vicențiu Ciorbaru
Sent: Thursday, 1 April 2021 09:59
To: Hollow Man
Cc: maria-developers
Subject: Re: [Maria-developers] GSOC21: MDEV-16375 &amp; MDEV-23143


&nbsp;

Hi Songlin!

It's great that you are excited about this project! Here are my thoughts on your proposal and what I think you should focus on:

JSON_NORMALIZE seems simple at first, but I believe there are a lot of corner cases. In order to get a proper specification for this function can you have a look at other databases, to see if they implement something similar? Have a look at the pandas python library, can you learn from their experience?


&nbsp;


Normalizing JSON can have some tricky cases such as:


a. How are arrays sorted if the values inside them are a mix of objects, arrays, literals, numbers.
b. How do you define a sorting criteria between two JSON objects in this case?
c. JSON is represented as text, however one can use it to store floating point values. How do you plan to compare doubles and how would those values be sorted? For example: 1 vs 1.0 vs 1.00 or 1000 vs 1e3?
d. What's the priority of null values, are they first, last?



The way we should handle this project is via TDD (Test Driven Development). You would first write your test cases, covering as many corner cases as possible, then implement the code such that it passes all the tests.

I suggest you add to your proposal some examples of how you define JSON_NORMALIZE and JSON_EQUALS to behave, so that we can see you have thought about points a, b, c, d from above.


&nbsp;


As for JSON_EQUALS, assuming JSON_NORMALIZE is done correctly, it may work as a simple strcmp between two normalized JSON objects, but I am not 100% confident at this point, you would have to prove it :)


&nbsp;


Vicențiu



&nbsp;

On Tue, 30 Mar 2021 at 09:00, Hollow Man <hollowman@xxxxxxxxxxxx&gt; wrote:


Hi community!


&nbsp;


I've had my proposal shared with https://drive.google.com/file/d/1sv0qbqt9W-ob3GqxygWwRGurpRS1lCiv/view&nbsp;, hope to get some feedback from the community.


&nbsp;


Songlin


&nbsp;


------------------ Original ------------------


From: &nbsp;"Hollow Man"<hollowman@xxxxxxxxxxxx&gt;;


Date: &nbsp;Thu, Mar 11, 2021 00:17 AM


To: &nbsp;"maria-developers"<maria-developers@xxxxxxxxxxxxxxxxxxx&gt;; 


Subject: &nbsp;GSOC21: MDEV-16375 &amp; MDEV-23143



&nbsp;


Hi MariaDB community!


&nbsp;


&nbsp; &nbsp;Glad to be here! My github account is @HollowMan6. Though I'm new to MariaDB community, I'm interested in MDEV-16375 &amp; MDEV-23143: Function to normalize a json value &amp; missing a JSON_EQUALS function for this year's GSOC project. Here are my first thoughts on these issues:


&nbsp;


&nbsp; &nbsp;I have checked part of the codebase and I think the two issues can be merged into one. First we can create a function named JSON_NORMALIZE to normalize the json, which automatically parses the inputed json document, recursively sorts the keys (for objects) / sorts the numbers (for arrays), removes the spaces, and then return the json document string.


&nbsp;


&nbsp; &nbsp;Then we create a function named JSON_EQUALS, which can be used to compare 2 json documents for equality realized by first seperately normalize the two json documents using JSON_NORMALIZE, then the 2 can be compared exactly as binary strings.


&nbsp;


&nbsp; &nbsp;I have taken some inspirations from the Item_func_json_keys and json_scan_start for parsing json documents, and I think it's possible to sort the keys using std::map in STL for objects.


&nbsp;


&nbsp; &nbsp;That's all for my ideas so far. Please correct me if I made some mistakes, and I'm going to work on my ideas later.


&nbsp;


Cheers!


&nbsp;


Hollow Man






_______________________________________________
Mailing list: https://launchpad.net/~maria-developers
Post to&nbsp; &nbsp; &nbsp;: maria-developers@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~maria-developers
More help&nbsp; &nbsp;: https://help.launchpad.net/ListHelp

&nbsp;