gtg-contributors team mailing list archive

[Lionel Dricot <ploum@xxxxxxxxx>]

Only for performances, could you please take the latest revision (658) of
my refactoring branch and try it with a lot of tasks and confirm me that
I'm not dreaming ?

I just can't believe what I'm seeing but that computer is fast.


Lionel

PS: tags display is still broken.



On Thu, 25 Feb 2010 01:27:16 +0100, Bertrand Rousseau
<bertrand.rousseau@xxxxxxxxx> wrote:
> On Wed, Feb 24, 2010 at 5:57 PM, Lionel Dricot <ploum@xxxxxxxxx> wrote:
>>
>> I've very good news for myself : I see the light.
>>
>> The FilteredTree is now working nearly perfectly. It is rock-solid,
even
>> when loading 300 tasks at the same time. I also implemented the Closed
>> Tasks pane with it. The FiltersBank is also working great.
> 
> Great news!
> 
>> Bad news : with a lot of tasks, the start time is really really really
>> slow. (like 10 times the current trunk launch time). On the other hand,
>> it
>> looks like other performance issues are mostly solved (like renaming a
>> subtask in a task).
> 
> Task renaming is maybe due to the fact that you disabled the refresh
> methods from the browser. Originally, it can from the fact that
> browser refresh is called at each letter you type by the editor. It
> should be worth looking at this too or it will probably show again
> sometime.
> 
>> So now, I need *you*. I need testing and profiling.
>>
>> Do : bzr branch lp:~gtg/gtg/gtg-refactor
>> and launch it first with debugging data then with actual data. Do some
>> profiling ( http://live.gnome.org/gtg/development ) and send me the
>> results.
> 
> Here's mine (on my netbook, so slownesses are sensible here):
> 
>    Ordered by: cumulative time
>    List reduced from 947 to 15 due to restriction <15>
> 
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.000    0.000   10.889   10.889
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
>         1    0.000    0.000   10.127   10.127
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
>         1    0.000    0.000    9.047    9.047
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
>         1    4.523    4.523    8.653    8.653 {gtk._gtk.main}
>       244    0.030    0.000    3.011    0.012
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1642(on_task_modified)
>       244    0.027    0.000    2.595    0.011
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:53(update_tags_for_task)
>      4032    0.263    0.000    2.457    0.001 {method 'row_changed' of
> 'gtk.TreeModel' objects}
>      7208    0.394    0.000    1.734    0.000 {method 'get_value' of
> 'gtk.TreeModel' objects}
>      7495    0.060    0.000    1.380    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:82(on_get_value)
>       732    0.023    0.000    1.347    0.002
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:788(tag_visible_func)
>       879    0.297    0.000    1.295    0.001
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/tagstore.py:305(get_tasks_nbr)
>     12660    0.107    0.000    1.140    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/requester.py:135(get_task)
>         1    0.000    0.000    1.074    1.074
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
>     12660    0.231    0.000    1.034    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:51(get_task)
>      2709    0.175    0.000    0.825    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:858(tag_sort_func)
> 
> As you can see, a lot of time is spent in update_tags_for_task. If you
> comment this out, get_value and on_task_modified don't even show up in
> the profiling log:
> 
>    Ordered by: cumulative time
>    List reduced from 944 to 15 due to restriction <15>
> 
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.000    0.000    9.428    9.428
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
>         1    0.000    0.000    8.639    8.639
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
>         1    0.000    0.000    7.443    7.443
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
>         1    5.892    5.892    7.012    7.012 {gtk._gtk.main}
>         1    0.001    0.001    1.173    1.173
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
>         1    0.000    0.000    0.776    0.776
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine)
>         1    0.002    0.002    0.775    0.775
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins)
>        12    0.002    0.000    0.587    0.049
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__)
>        12    0.001    0.000    0.544    0.045
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module)
>        12    0.003    0.000    0.514    0.043 {imp.load_module}
>        29    0.003    0.000    0.442    0.015
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml)
>         2    0.000    0.000    0.430    0.215
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore)
>        10    0.356    0.036    0.395    0.040 {method 'show' of
> 'gtk.Widget' objects}
>       975    0.064    0.000    0.383    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tasktree.py:94(on_get_value)
>        29    0.001    0.000    0.334    0.012
> /usr/lib/python2.6/xml/dom/minidom.py:47(toprettyxml)
> 
> It's probably not the task of the browser to refresh each tag related
> to a task. Since Tags have references to their tasks, they are
> modified as well when a Task is edited. So we should probably let the
> tag emit a signal to warn everyone they changed (much like the Task
> are doing now). Let the objets which care about it to register
> themselve to this signal through the requester.
> 
> That would imply having a "on_tag_modified" in the browser that would
> trigger some job to perform on the browser (probably not much since,
> if I understood well, most of the filtering work is now done outside
> the browser)
> 
>> If you want to contribute, there's a huge cleaning to do in the
>> browser.py
>> ! A lot of cruft should be removed (and I don't understand everything
>> there).
> 
> I agree with what you say, but it would be more efficient if you could
> be more specific and point out some of these cruft.
> 
>>
>> What does not work :
>> - tag filtering. They are implemented.  We just need to do
>> "self.req.apply_filter(tagname)" when a particular tag is selected but
I
>> don't know where it is in browser.py and I'm a bit tired. (any help is
>> welcome here)
>> - drag-n-drop : code is commented out in tasktree.py. Maybe it will
work
>> without any trouble.
>> - multiparent (not tested yet)
>> - tag count in tagtree
>>
>> What is broken :
>> - plugins and plugin_api
>> - dbus interface
>>
>>
>> I was expecting better performances but FilteredTree appeared to be a
lot
>> more complex than expected. So now, it's time to optimize it a bit and
>> I'm
>> wondering where is the bottleneck.
> 
> Like I wrote above, it seems to be situated around tags update
> mechanisms and TagTreeModel updates.
> 
>>
>> Thanks for listenning,
>>
>>
>> Lionel
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~gtg-contributors
>> Post to     : gtg-contributors@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~gtg-contributors
>> More help   : https://help.launchpad.net/ListHelp
>>