u1db-discuss team mailing list archive

[Stuart Langridge <stuart.langridge@xxxxxxxxxxxxx>]

On Tue, 2011-11-15 at 20:36 +0100, Mikkel Kamstrup Erlandsen wrote:
[snip a bunch of stuff that I hope I answered in the previous reply]
> > 5. How data is actually stored in a backend is backend-specific. The
> > get_doc function has to return *something* -- if it's returning a native
> > object, then the u1db layer still has to read data out of the back end
> > and turn it into that native object (a back end is unlikely to store an
> > *actual native object* for retrieval), so there's always some sort of
> > parsing step involved
> >
> All the more reason to hide the storage format from the API as well I'd say.
> 
> The only place where I'd say one could need access to the raw storage 
> format would be to implement some optimized parsing routines that does 
> not require allocating full objects all over the place - ala SAX parsing 
> fx. This could be added at a later point to the API though by using some 
> visitor pattern or other though, so not really an argument.
> 

JSON is not the storage format.

The storage format is completely defined by the back end. Your back end
can store the data in a big csv file if it likes. We're not talking
about JSON because that's the format the data is stored in. We're
talking about JSON because that's the format your implementation has to
talk in order to do syncs. So you have to have a thing which translates
documents from on-the-disk format (sqlite, csv, whatever) to JSON
*regardless of your storage format*, so that you can do syncs. The
question is: is it better to have *two* translators (one from
on-disk-format to JSON for syncs, and another from on-disk-format to
native objects for API users), even though that's extra work? On
balance, I think it is, as suggested by lucio, but deciding that is the
point of this discussion :-)

sil