u1db-discuss team mailing list archive

[Jason Gerard DeRose <jderose@xxxxxxxxxxx>]

Hello all,

Before I dive into the U1DB design, I wanted to share a brain dump of my
experience working with desktopcouch and CouchDB full-time for over a year,
strengths and weaknesses that have stood out in both.

Quick background: both Novacut <https://launchpad.net/novacut> (video
editor) and Dmedia <https://launchpad.net/dmedia> (distributed media
library, foundation of Novacut) store data in CouchDB and heavily utilize
CouchDB's wonderful sync properties.

Novacut is a fairly easy problem to model, all in all.  The documents
describing a simple Novacut edit look like
this.<http://paste.ubuntu.com/661795/>
Dmedia is an unbelievably difficult and painful problem to model
(seriously, it makes my head hurt).  The document describing a single file
in Dmedia looks like this. <http://paste.ubuntu.com/754524/>  But both
proved to be far easier to model using a document oriented database.

*CouchDB: The Good*

   - *The data model* - CouchDB nailed this and although there are a few
   subtle things I think I'd do differently, I'd want basically the same thing
   - *Built in layers* - the data model is what enables the sync properties
   (including real-time sync), but different approaches can be taken at the
   sync layer pretty easily while keeping the data model the same
   - *Simple, focused core API* - the CouchDB REST API is one of the best
   APIs I've ever used; all the CouchDB "wrappers" I've worked with were both
   more complicated and less expressive (desktopcouch included, sorry!); I
   highly recommend that U1DB adopt an API with pretty much the exact same
   REST semantics (regardless whether HTTP is involved at a given layer)
   - *JSON documents* - purely cloud based apps suck because then the user
   experience hinges on Internet reliability; purely native/local apps suck
   because then the app is cut off from other devices and perhaps more
   importantly, other people... I think there is room for huge innovation in
   building apps that thrive in both worlds, and desktopcouch was certainly
   the bold first mover here; JSON seems an absolute must for bridging the two
   worlds

*CouchDB: The Bad*

   - *Replicating views, storing views in the DB* - from the desktop
   perspective, views are app data, not user data, so get them out of the
   user's database! different devices will have different apps installed, and
   different versions at that; plus in my experience, it seems easier to
   stabilize the schema than the views... so separate them!
   - *The implementation* - I have a profound respect for the design of
   CouchDB, but I can't help feel it's a brilliant design with a lackluster
   implementation; sync is *the* killer CouchDB feature, yet the replicator
   is horribly unreliable (including in my own tests outside of Ubuntu One); I
   feel Erlang is rather to blame here... I think they would have been better
   working with a language more people know, that was faster to develop with
   and especially faster to write unit tests with
   - *Erlang* - as my interest is using CouchDB on the desktop and on
   mobile devices, I consider Erlang a huge liability... it means dragging in
   an entire runtime that wouldn't be needed otherwise, it's poor serial
   performance is a bad fit for the desktop, and it can't easily integrate
   with the platform (DBus, etc)
   - *Unfocused at the periphery* - I think the CouchDB folks perhaps got a
   bit sucked into adding trendy ways to incorporate JavaScript, stuff at the
   periphery that really belongs in separate projects

*desktopcouch: The Good*

   - *The Vision* - providing structured data sync at the platform level is
   bold and disruptive and important... lets have more of that! But honestly,
   Novacut was truly inspired by desktopcouch... it was learning about
   desktopcouch that made me realize that CouchDB could solve the hardest
   parts of this crazy collaborative editor I was daydreaming about.

*desktopcouch: The Bad*

   - *API too high level, too restrictive* - desktopcouch would have been
   better off exposing the CouchDB REST API semantics without any wrappings,
   although it took me a lot of app development time to realize this; trying
   to enforce things like the "record_type" was a bad move, IMHO... in my
   apps, it didn't solve any problems, but it did make things more
   cumbersome... give app developers freedom here, they'll figure out what
   works best for their problems
   - *Too monolithic* - although hard to avoid when blazing a new trail,
   there is still a lesson to be learned, i think... particularly that
   whatever manages sync should be a separate component
   - *Tried to freeze API too early* - I know there were nothing but good
   intentions behind this, but having a great API is more important than
   maintaining backward compatibility, especially early on... your early
   adopter devs will happily chase a changing API if it's improving... and
   please utilize feedback from your early adopters!

Alright, I'll starting digging into U1DB and chime in later with my
thoughts!

Cheers,

Jason