u1db-discuss team mailing list archive

[Rodney Dawes <dobey.pwns@xxxxxxxxx>]

On Tue, 2011-11-15 at 19:04 +0000, Stuart Langridge wrote:
> I know jam has been profiling this stuff, to actually confirm whether
> JSON is slower or not. For now, I don't want to address speed; I just
> want to summarise both approaches with pros and cons of each to make
> sure we're all talking about the same thing.

For clarity, I have not been talking about performance at all. I have
only been talking about usability (IE, making it easier for app
developers to actually use this stuff). I could care less to have the
performance arguments right now, as there is very little we're going to
gain by picking one format over another, to store in SQLite anyway. The
real gains are going to be made in how we manage in-memory management of
the documents, and how often we have to hit disk/network.


> 1. functions returning/taking JSON strings means that the API
> documentation is basically identical across platforms. This makes
> documentation of u1db easier (the documentation applies whether you're
> using a Python port, a Ruby port, a C port, etc)

This is just nonsense. Language differences are already going to
dictate there be different docs for different languages, anyway. The
current implementation in Python makes a very grand example of this,
by returning various things in different ways. Having a native object
for a Document, will actually make it easier to have one piece of
documentation across multiple implementations.

> 2. The sync function uses JSON to talk between u1db servers[1]. This
> means that a u1db implementation needs to be able to serialise a stored
> document to JSON and deserialise it from JSON into storage anyway.
> Adding a separate native document type means that there's extra coding

This is also not quite true. The storage format should be irrelevant to
the API being exposed to developers. Passing around JSON strings, is
basically the equivalent of forcing the developer to deal with the
threading issues that SQLite creates. As U1DB is a library which we
expect developers to use in their applications, then we should make it
as easy as possible, and hide as much pointless implementation detail
as possible. As an app developer, I don't want to have to muck about
with JSON parsers and generators. I want nice high-level objects which
make my life easy. The tiny bit of extra code to make this happen in
U1DB is basically irrelevant, and not a good argument.

> 3. The test suite will test compliance of a u1db version by talking to
> the sync server. If the syncing uses JSON but the use in-app is using
> something else (native objects) then the test suite will not be actually
> testing a common code path, which makes it harder to establish whether
> an implementation is compliant

Is there a test suite which is NOT Python unit tests yet, that can run
over arbitrary code in arbitrary languages? Or a definition of what
"compliance" actually means? Making this work the way you would like, is
probably several orders of magnitude more difficult than providing a
nice OO API for people to use.

> 4. (a minor objection at best) it's not clear that there is a sensible
> "native object" for some implementations (e.g., C). However, a "pure C"
> implementation might be useless for this reason, suggesting that there'd
> be a "C with glib" implementation and this would have an obvious native
> object type.

The "native object" in C is a struct[1]. Presumably we'd use json-c for
a plain ANSI C implementation. For "C with glib" implementation, I would
expect the Vala implementation I've started working on, would be the
thing to use. And it is exactly why I have brought this concern up. :)


[1] https://github.com/jehiah/json-c/blob/master/json_object.h