← Back to team overview

zim-wiki team mailing list archive

Re: Online storage


Pedro wrote:
... 8< ..
Jaap, can you jump in the discussion and drop a quick briefing of the basics on zim data storage?

The storage is abstracted by the "store" classes. The default file storage does a straightforward mapping from a page name to a file name and saves data in one file per page.

By writing a new store class you can e.g. change the layout of the notebook directory or use a completely different backend like e.g. IMAP to store data.

The API for the store classes is pretty dump. They can get a page object and store it again and provide page listings per namespace. The 'store_page' method for the backend is called every time the page is saved - so not only when navigating away, but also when autosaving on 5sec intervals.

If you are mainly interested in using an online storage for synchronization purposes, I see the following approaches:

0) Do nothing and let the file system layer handle it. This is e.g. the case when you use UbuntuOne to store a zim notebook. You just put the notebook in the synced folder and that is all.

1) Write a store backend for a specific protocol like IMAP or . This has it's advantages, but one disadvantage is that you can not touch your notes when you are offline. Another disadvantage is that this means a certain amount of custom code to make zim speak a specific protocol. Biggest advantage from user point of view is that because the server can have a single state you will almost never have merging conflicts.

2) Or we can treat merging in parallel to the actual storage, only assuming that the storage is done on a local file system. This approach is similar to how versioning is handled. This means we use available solutions for synchronizing files. The advantage is that this way we re-use existing code and have to do minimal zim specific work. My preferred solution would be to use a versioning system like bazaar or subversion to do the synchronization, but we could hook up rsync in much the same way.

The main thing we need is a signal that triggers synchronization. Doing this at every auto-save is too often. But we could trigger e.g. at start of the application, when navigating away from a page, when closing the application etc. Or even apply a time-based interval when zim is running.

The synchronization would check for new versions on the server and push any modifications we made. Only disadvantage is that we need to take into account the possibility of merging conflicts. My proposal would be to try syncing automatically and when a conflict is found prompt the user to either do a manual merge or keep working (in the later case disable the event, till some later time, e.g. next application close / start and prompt again).

If we define the conditions when synchronisation should be triggered various plugins could handle the specific systems. E.g. the version control plugin could provide this functionality, but there could be a separate rsync plugin that does the same.

Btw. I'm working now to add version control to the python version - expect to release that update either thursday or sunday. So there is some actual working code to at least support bazaar, and this can easily be extended to other systems like svn, git, etc.

Hope this provides the background you were looking for.