u1db-discuss team mailing list archive

[Stuart Langridge <stuart.langridge@xxxxxxxxxxxxx>]

On 17/07/12 11:52, Eric Casteleijn wrote:
> > And here's the 4 view functions, respectively, that drive these 4 tasks:
> >
> > file_stored = """
> > function(doc) {
> >      if (doc.type == 'dmedia/file') {
> >          var key;
> >          for (key in doc.stored) {
> >              emit(key, null);
> >          }
> >      }
> > }
> > """
> The "stored" dictionary was the hardest thing to get right. Hopefully
>
> This one's not currently supported but could easily be if a 'keys' 
> index mapping were to be added, and then the index would be:
>
> doc.type, keys(doc.stored)
Me, if I were storing this stuff in u1db, I'd make doc.stored be a list:

{
    "_id": "WEFB4GYQBWOEGCUA5AZEXOKWHCB4IUSYZ2TGJBJUPFWTWCNL",
    ...
    "stored": [
        {
            "store_id": "3KX6WX6YAUQIQ7M2Y3BMPSAI",
            "copies": 1,
            "mtime": 1338564292,
            "verified": 1338571742
        },
        {
            "store_id": "PSRJGX2O2N77XLBEBI2TNDNU",
            "copies": 1,
            "mtime": 1338560150,
            "verified": 1338564292
        },
        {
            "store_id": "YE4SYZAVTFEBDI2JZQBUTYQY",
            "copies": 1,
            "mtime": 1338560150
        }
    ]
}


and then this index would be: db.create_index("stored", "type", 
"stored.store_id").

This doesn't currently work because we don't index a list of dicts, but 
we've planned to right since the beginning, it just hasn't been 
implemented yet :)

> > file_verified = """
> > function(doc) {
> >      if (doc.type == 'dmedia/file') {
> >          var key;
> >          for (key in doc.stored) {
> >              emit([key, doc.stored[key].verified], null);
> >          }
> >      }
> > }
> > """
Similarly, with doc.stored as a list, this becomes
db.create_index('verified', 'type', 'stored.verified', 'stored.store_id')

Doesn't work because ditto. Also, index ordering is a bit underspecified 
right now, but this will order by verified then by storeid. Swap to 
order by storeid then verified (if it's best to check everything in a 
store at once, rather than all unverified and old ones first regardless 
of store).

As Eric says, the latter two are harder, because we don't, currently, 
have a way of combining multiple index expression results (sort of 
analogously to SQL's aggregate functions). Naive thought:

db.create_index('fragile', 'type', 'origin', 'number(sum(stored.copies), 
5)')
and then
fragile_files = db.get_range_from_index('fragile', 'dmedia/file', 
'user', ('00000', '00003'))

This won't work because we don't currently have a transformation 
function sum(), but it (and count()) wouldn't be too hard to add. We'd 
need to think through the ramifications, though (what if you aggregate 
on the second field of a three-field index expression? etc)

sil