u1db-discuss team mailing list archive

[Stuart Langridge <stuart.langridge@xxxxxxxxxxxxx>]

On Tue, 2011-11-15 at 14:36 -0500, Rodney Dawes wrote:
> > 1. functions returning/taking JSON strings means that the API
> > documentation is basically identical across platforms. This makes
> > documentation of u1db easier (the documentation applies whether you're
> > using a Python port, a Ruby port, a C port, etc)
> 
> This is just nonsense. Language differences are already going to
> dictate there be different docs for different languages, anyway. The
> current implementation in Python makes a very grand example of this,
> by returning various things in different ways. Having a native object
> for a Document, will actually make it easier to have one piece of
> documentation across multiple implementations.

Yes, there will obviously be some small differences between languages
anyway. However, there's a large benefit to having the calls look as
similar as possible between languages. A big part of the u1db-using
community will be people who want to sync data across different devices
(since it's a syncable DB) -- obviously some of those will be (say)
Ubuntu-to-Ubuntu syncs, but another large part of that will be syncing
across devices which will likely be using different U1DB
implementations; having them be able to easily transfer the knowledge
from one to the other is very helpful.

As Lucio suggests, maybe we say "get_doc returns a Document object, and
what a Document object looks like is implementation-defined", but I'd
only be happy with that if a Document object looks as similar as
possible between implementations; is that reasonable? For example, given
a Document object, is it sensible to look at
docobj["field1"]["subfield"] across implementations? (Substitute
"get-a-mapping-field" syntax for each language, of course, so
docobj->field1->subfield for PHP or whatever.)

> > 2. The sync function uses JSON to talk between u1db servers[1]. This
> > means that a u1db implementation needs to be able to serialise a stored
> > document to JSON and deserialise it from JSON into storage anyway.
> > Adding a separate native document type means that there's extra coding
> 
> This is also not quite true. The storage format should be irrelevant to
> the API being exposed to developers. Passing around JSON strings, is
> basically the equivalent of forcing the developer to deal with the
> threading issues that SQLite creates. As U1DB is a library which we
> expect developers to use in their applications, then we should make it
> as easy as possible, and hide as much pointless implementation detail
> as possible. As an app developer, I don't want to have to muck about
> with JSON parsers and generators. I want nice high-level objects which
> make my life easy. The tiny bit of extra code to make this happen in
> U1DB is basically irrelevant, and not a good argument.

The storage format is irrelevant, I agree. However, this isn't about the
storage format. Every implementation has to be able to generate and
parse JSON so that it can sync. This particular point is purely about
how everyone has to do that *anyway* and therefore if we use JSON in-app
then the u1db implementor doesn't need to make a native object as well.
I agree entirely that a native object is easier for app developers to
deal with!

> > 3. The test suite will test compliance of a u1db version by talking to
> > the sync server. If the syncing uses JSON but the use in-app is using
> > something else (native objects) then the test suite will not be actually
> > testing a common code path, which makes it harder to establish whether
> > an implementation is compliant
> 
> Is there a test suite which is NOT Python unit tests yet, that can run
> over arbitrary code in arbitrary languages? Or a definition of what
> "compliance" actually means? Making this work the way you would like, is
> probably several orders of magnitude more difficult than providing a
> nice OO API for people to use.

There is no test suite which works across arbitrary code in arbitrary
languages (nor is such a thing sanely possible). That's why the test
suite will talk to the HTTP server version of your implementation. This
test suite doesn't exist yet, but it will do; if you want your
implementation tested (and you should do) then you have to implement the
code required to make your implementation able to run as a server. (This
code should not be hard - you don't have to be able to cope with high
loads on this server, for example - but you'll have to do it to get the
test suite, and we want to encourage people to run the test suite!)

sil