zim-wiki team mailing list archive

[Jaap Karssenberg <jaap.karssenberg@xxxxxxxxx>]

Pedro wrote:
... 8< ..
> Jaap, can you jump in the discussion and drop a quick briefing of the 
> basics on zim data storage?

The storage is abstracted by the "store" classes. The default file 
storage does a straightforward mapping from a page name to a file name 
and saves data in one file per page.

By writing a new store class you can e.g. change the layout of the 
notebook directory or use a completely different backend like e.g. IMAP 
to store data.

The API for the store classes is pretty dump. They can get a page object 
and store it again and provide page listings per namespace. The 
'store_page' method for the backend is called every time the page is 
saved - so not only when navigating away, but also when autosaving on 
5sec intervals.

If you are mainly interested in using an online storage for 
synchronization purposes, I see the following approaches:

0) Do nothing and let the file system layer handle it. This is e.g. the 
case when you use UbuntuOne to store a zim notebook. You just put the 
notebook in the synced folder and that is all.

1) Write a store backend for a specific protocol like IMAP or . This has 
it's advantages, but one disadvantage is that you can not touch your 
notes when you are offline. Another disadvantage is that this means a 
certain amount of custom code to make zim speak a specific protocol. 
Biggest advantage from user point of view is that because the server can 
have a single state you will almost never have merging conflicts.

2) Or we can treat merging in parallel to the actual storage, only 
assuming that the storage is done on a local file system. This approach 
is similar to how versioning is handled. This means we use available 
solutions for synchronizing files. The advantage is that this way we 
re-use existing code and have to do minimal zim specific work. My 
preferred solution would be to use a versioning system like bazaar or 
subversion to do the synchronization, but we could hook up rsync in much 
the same way.

The main thing we need is a signal that triggers synchronization. Doing 
this at every auto-save is too often. But we could trigger e.g. at start 
of the application, when navigating away from a page, when closing the 
application etc. Or even apply a time-based interval when zim is running.

The synchronization would check for new versions on the server and push 
any modifications we made. Only disadvantage is that we need to take 
into account the possibility of merging conflicts. My proposal would be 
to try syncing automatically and when a conflict is found prompt the 
user to either do a manual merge or keep working (in the later case 
disable the event, till some later time, e.g. next application close / 
start and prompt again).

If we define the conditions when synchronisation should be triggered 
various plugins could handle the specific systems. E.g. the version 
control plugin could provide this functionality, but there could be a 
separate rsync plugin that does the same.

Btw. I'm working now to add version control to the python version - 
expect to release that update either thursday or sunday. So there is 
some actual working code to at least support bazaar, and this can easily 
be extended to other systems like svn, git, etc.

Hope this provides the background you were looking for.

Regards,

Jaap