tomdroid-dev team mailing list archive

[Cornelius Hald <hald@xxxxxxxxx>]

Hi Oliver,

I don't have Tomboy developer experience, but I came across similar
problems while developing a Tomboy client for Maemo, so maybe my input
could help you as well.

Normally we store the data in xml files, just like Tomboy is doing it,
but there is also a backend plugin that saves the notes directly into a
Midgard database.

So what I'm doing is storing the <note-content> subtree as string in a
member of each Note object. When showing a note only this subtree is
parsed and displayed. When saving a note object to the database we only
store this subtree in the database as xml, the metadata is saved
directly.
When saving to a xml file, the rest of the file is constructed and the
<note-content> subtree is merged with it.

Getting the sub tree was no problem here, but we use a different parser
model not SAX. I guess doing all the work with SAX is rather a big pain.
I'm using XmlTextReader from libxml2. I think there is also an XmlReader
interface on Android. Maybe that will make things easier?

For me the problem was the parsing of the <note-content> sub tree,
because it has no namespace definitions after extracting it from the
main document. They are only available on the <note> element and libxml
throws an error if you try to parse an element like <size:big> without
having a reference to the "size" namespace.

So what I had to do was to append the namespace on the fly to the
<note-content> element and later when merging back with the rest of the
xml file I had to remove the redundant namespaces again.

But that probably also depends on the parser which you are using.

For your solution #1 I don't see a big problem there. Only you need to
decide which metadata is the "correct" one. The one in the database or
the one on in the xml document. It's not really nice to store redundant
data, but in this case it's probably not too bad and the extra cost of
parsing through the metadata shoudn't be that big.

About solution #2. Well, I think we all know that using regexp on xml is
not the right thing (tm) to do. But I don't see a big problem in this
case as the structure of those documents is well know and there
shouldn't be any surprises. In fact I'm doing some regexp suff on xml in
my code as well. Not proud of it, be it works ;)

BTW, does using a database for storage really improves speed for you?
I'm just asking because I had no performance issues so far. But only
tested with around 100 ntoes. I can load them in about 1 second on the
Nokia N810 which is a quite slow device. But your results would really
interest me. Are you using sqlite or something else?


I'm not sure if that does help you somehow. Hopefully I didn't miss your
point completely ;)

Cheers!
Conny



On Tue, 2009-11-17 at 00:17 -0500, Olivier Bilodeau wrote:
> hi guys,
> 
> Just working on reviewing some code for tomdroid and it got me
> thinking about something I would like to ask people with tomboy
> experience.
> 
> Let me explain quickly.
> 
> There is this hack we do right now in the storage-redesign branch[1]
> to re-create XML tags after we parse them in a note. The deal is we
> want to parse the xml tags except the ones in
> <note-content>..</note-content> so we store XML in our sqlite database
> for the note-content portion and parse it again on the fly when a Note
> is viewed.
> 
> I _really_ find it disgusting, it looks like:
> 
> String tag = "<";
> 
> if (uri != null) {
>     if (uri.equals(NS_LINK)) {
>         tag += PREFIX_LINK+":";
>     } else if (uri.equals(NS_SIZE)) {
>         tag += PREFIX_SIZE+":";
>     }
> }
> 
> tag += localName+">";
> xmlContent.append(tag);
> 
> for open tags and close ones inside of a focused xml class.
> 
> Now, (and here is where I need your experience), I was thinking about
> 2 potentials solutions here (aside from the fact that we might just be
> misusing Java's XML parsing lib):
> 
> 1- store the whole note instead of just note-content in the database
> This way we won't lose addins metatags, full note encryption support
> will be possible, etc. Also, parsing the few lines of metadata on each
> load won't be that big of a penalty since its the note's content that
> is always the biggest.
> 
> 2- Forget about xml parsing just regexp out the note-content portion
> then parse the rest of the note. Or just use an xml lib that allows to
> extract part of the raw xml structure (or find the way to do it in
> Sax).
> 
> I prefer #1 but I'm thinking maybe there are major roadblocks to this
> solution that I don't foresee.
> 
> What does the tomboy people think?
> 
> If I'm not clear enough, ask questions i'll readjust.
> 
> Thanks for your time.
> -- 
> Olivier Bilodeau <olivier@xxxxxxxxxxxxxxxxx>
> _______________________________________________
> Tomboy-list mailing list
> Tomboy-list@xxxxxxxxxxxxxxxxxxxxxxxxx
> http://lists.beatniksoftware.com/listinfo.cgi/tomboy-list-beatniksoftware.com