zeitgeist team mailing list archive

[Seif Lotfy <639737@xxxxxxxxxxxxxxxxxx>]

Here a more detailed benchmarking:

1) NO CHANGES: ~ 0.12 s
------------------------
DEBUG:zeitgeist.engine:Inserted 1 events in 0.107999s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.155844s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.128242s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.161610s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.106275s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.098261s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.112802s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.095572s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.112714s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.151126s


2) PRAGMA locking_mode = EXCLUSIVE: ~ 0.104 s
-----------------------------------------------------
DEBUG:zeitgeist.engine:Inserted 1 events in 0.102811s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.111825s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.099751s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.112921s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.104319s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.116963s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.101241s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.098185s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.101261s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.098256s

3) PRAGMA synchronous =NORMAL: ~ 0.098 s
-----------------------------------------------
DEBUG:zeitgeist.engine:Inserted 1 events in 0.089371s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.092805s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.075517s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.102996s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.080430s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.122163s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.088753s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.151342s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.099504s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.082245s

4) PRAGMA locking_mode = EXCLUSIVE && PRAGMA synchronous =NORMAL: ~ 0.912 s
------------------------------------------------------------------------
DEBUG:zeitgeist.engine:Inserted 1 events in 0.083371s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.076682s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.075443s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.091788s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.076546s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.099368s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.088096s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.091075s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.088003s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.146164s

5) PRAGMA journal_mode = WAL ~ 0.071 s
------------------------------------------
DEBUG:zeitgeist.engine:Inserted 1 events in 0.072993s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.068750s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.067910s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.066470s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.075882s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.071495s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.114053s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.055614s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.059923s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.063065s

6) ALL TOGETHER NOW: ~ 0.001 s
---------------------------------
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000850s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000790s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.001318s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.001316s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000733s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.001196s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000973s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000729s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000896s
DEBUG:zeitgeist.engine:Inserted 1 events in 0.000887s

I think the ALL combination is a good go for the desktop. However WAL is
only supported by 3.7 SQLite which is not in Mameo yet and only in
testing in Debian. However the other non wall changes could help us
reduce the insertion time on the Maemo by 50%

-- 
Improve insertion times
https://bugs.launchpad.net/bugs/639737
You received this bug notification because you are a member of Zeitgeist
Framework Team, which is subscribed to Zeitgeist Framework.

Status in Zeitgeist Framework: Triaged

Bug description:
We insert pretty slowly with an average of 0.15 seconds for one event on my core i5 2.5 GHz beast.

RainCT had some optimization possibilities:
1) PRAGMA synchronous=OFF
2) PRAGMA journal_mode=OFF

The Chat:
------------------------------------------------------------
<kamstrup> I think we are - but I can't recall... in case of failed transactions - but I don't even know if we use transactions these days...
<seif> <RainCT> try synchronous=OFF
<seif> <RainCT> but it can corrupt your database if your phone dies while ZG is inserting
<seif> <RainCT> and journal_mode=MEMORY
<seif> <RainCT> or OFF since we don't use rollback anyway
<seif> so maybe journal_mode = OFF is a good start?
<kamstrup> okay, he's probably right...
<kamstrup> 'grep -Ri rollback _zeitgeist/' is your friend :-)
<kamstrup> apparently we are not using rollback...

More info can be found here: http://www.sqlite.org/pragma.html
------------------------------------------------------------

In order to get a better picture of what's going on, can you please try to get some more information, like:
1) How many events are in your database?
2) What's the insertion time for one event into an empty db?
3) Out of this 0.15 secs, how many time is spend in our python code, and what's the time of the actual sql action?
4) How much faster is adding 10 events at once compared to adding them one at a time?
5) You think 0.15 secs is slow for inserting one event, what time do you expect, and why?