u1db-discuss team mailing list archive

[John Meinel <john@xxxxxxxxxxxxxxxxx>]

It does seem like a place where user defined funcs could work. (Here are
some documents, give us a string to index on). Now whether the complexity
of supporting user defined functions is worth the effort..

John
=:->
On Jul 17, 2012 5:52 AM, "Eric Casteleijn" <eric.casteleijn@xxxxxxxxxxxxx>
wrote:

>
> Hi Jason,
>
> I thought it might be instructive (for me ;) to try and see to what extent
> those views could be implemented with the current state of the art of u1db.
> Note that I probably will get stuck somewhere, as u1db
> indexes are much much simpler than couchdb views.
>
>  Each file in Dmedia has a corresponding doc in CouchDB. This is an
>> example doc with the essential schema that drives all the automation
>> behaviors (the "make file management go away" aspect of Dmedia):
>>
>> {
>>      "_id": "**WEFB4GYQBWOEGCUA5AZEXOKWHCB4IU**SYZ2TGJBJUPFWTWCNL",
>>      "type": "dmedia/file",
>>      "time": 1338560151.467609,
>>      "atime": 1342088785,
>>      "bytes": 26566829,
>>      "origin": "user",
>>      "stored": {
>>          "3KX6WX6YAUQIQ7M2Y3BMPSAI": {
>>              "copies": 1,
>>              "mtime": 1338564292,
>>              "verified": 1338571742
>>          },
>>          "PSRJGX2O2N77XLBEBI2TNDNU": {
>>              "copies": 1,
>>              "mtime": 1338560150,
>>              "verified": 1338564292
>>          },
>>          "YE4SYZAVTFEBDI2JZQBUTYQY": {
>>              "copies": 1,
>>              "mtime": 1338560150
>>          }
>>      }
>> }
>>
>> This particular file is stored in 3 different "stores" (let's say hard
>> drives, but a store could also be something like UbuntuOne or S3).
>> Each of these stores has a durability confidence of 1 copy, and the
>> total durability for this file is 3 copies. And the copy in the last
>> store hasn't yet been verified (full content-hash verification).
>>
>> The "stored" dictionary was the hardest thing to get right. Hopefully
>> And here's the 4 view functions, respectively, that drive these 4 tasks:
>>
>> file_stored = """
>> function(doc) {
>>      if (doc.type == 'dmedia/file') {
>>          var key;
>>          for (key in doc.stored) {
>>              emit(key, null);
>>          }
>>      }
>> }
>> """
>>
>
> This one's not currently supported but could easily be if a 'keys' index
> mapping were to be added, and then the index would be:
>
> doc.type, keys(doc.stored)
>
>
>  file_verified = """
>> function(doc) {
>>      if (doc.type == 'dmedia/file') {
>>          var key;
>>          for (key in doc.stored) {
>>              emit([key, doc.stored[key].verified], null);
>>          }
>>      }
>> }
>> """
>>
>
> this one is harder, as we don't have a way to peek into variable parts of
> the document, and it's not immediately obvious to me how that could be
> added without adding a great deal of complexity, so this might just be a
> use case that u1db is too simple for. (Restructuring the document to
> contain a list of the keys somewhere would work, but that's assuming you
> can and want to do that.)
>
>  file_fragile = """
>> function(doc) {
>>      if (doc.type == 'dmedia/file' && doc.origin == 'user') {
>>          var copies = 0;
>>          var key;
>>          for (key in doc.stored) {
>>              copies += doc.stored[key].copies;
>>          }
>>          if (copies < 3) {
>>              emit(copies, null);
>>          }
>>      }
>> }
>> """
>>
>
> Same for this one. We intentionally didn't support any kind of real logic
> in the views, to reduce complexity, both for view authors, and for the
> indexing engine.
>
>  file_reclaimable = """
>> function(doc) {
>>      if (doc.type == 'dmedia/file' && doc.origin == 'user') {
>>          var copies = 0;
>>          var key;
>>          for (key in doc.stored) {
>>              copies += doc.stored[key].copies;
>>          }
>>          if (copies >= 3) {
>>              for (key in doc.stored) {
>>                  if (copies - doc.stored[key].copies >= 3) {
>>                      emit([key, doc.atime], null);
>>                  }
>>              }
>>          }
>>      }
>> }
>> """
>>
>
> And same for this one.
>
> --
> eric casteleijn
> https://launchpad.net/~**thisfred <https://launchpad.net/~thisfred>
>
>
>
>
> --
> Mailing list: https://launchpad.net/~u1db-**discuss<https://launchpad.net/~u1db-discuss>
> Post to     : u1db-discuss@lists.launchpad.**net<u1db-discuss@xxxxxxxxxxxxxxxxxxx>
> Unsubscribe : https://launchpad.net/~u1db-**discuss<https://launchpad.net/~u1db-discuss>
> More help   : https://help.launchpad.net/**ListHelp<https://help.launchpad.net/ListHelp>
>