gtg-contributors team mailing list archive

[Lionel Dricot <ploum@xxxxxxxxx>]

Thanks for your input. I didn't touch anything related to tags but,
indeed, it seems that it is sometimes broken. After some test, it appears
that the task view are, in fact, quite efficient. I did some very very
basic optimisations yesterday and I'm currently between 100% and 200% of
the trunk loading time (when I wrote the mail yesterday, we were at 1000%
so it looks like the concept is good).

The problem is that taskbrowser was the root of everything. Everything was
in it, from filtering tasks/tags to windows management and tasks updating.

Every filtering feature should be moved to the filterbank. I will create a
new module, called view_manager, that will contains the current manager.py
in core and will be responsible of loading the preferences and launching
the gtk loop.

Also, it looks like the tag store itself might benefit from some
refactoring and some love. 


Bertrand : I removed the refresh because it's not needed anymore. That's
the beauty of all of that. The only global refreshes now happen when a
filter is applied/unapplied. We will even be able to reduce the artificial
7 second delay before updating that we introduced in the TaskEditor.


PS : I'm quite bad at undestanding profiling output, so thanks for your
interpretation. I remember that you found a visualisation tool. What was
the name again ?


On Thu, 25 Feb 2010 01:27:16 +0100, Bertrand Rousseau
<bertrand.rousseau@xxxxxxxxx> wrote:
> On Wed, Feb 24, 2010 at 5:57 PM, Lionel Dricot <ploum@xxxxxxxxx> wrote:
>>
>> I've very good news for myself : I see the light.
>>
>> The FilteredTree is now working nearly perfectly. It is rock-solid,
even
>> when loading 300 tasks at the same time. I also implemented the Closed
>> Tasks pane with it. The FiltersBank is also working great.
> 
> Great news!
> 
>> Bad news : with a lot of tasks, the start time is really really really
>> slow. (like 10 times the current trunk launch time). On the other hand,
>> it
>> looks like other performance issues are mostly solved (like renaming a
>> subtask in a task).
> 
> Task renaming is maybe due to the fact that you disabled the refresh
> methods from the browser. Originally, it can from the fact that
> browser refresh is called at each letter you type by the editor. It
> should be worth looking at this too or it will probably show again
> sometime.
> 
>> So now, I need *you*. I need testing and profiling.
>>
>> Do : bzr branch lp:~gtg/gtg/gtg-refactor
>> and launch it first with debugging data then with actual data. Do some
>> profiling ( http://live.gnome.org/gtg/development ) and send me the
>> results.
> 
> Here's mine (on my netbook, so slownesses are sensible here):
> 
>    Ordered by: cumulative time
>    List reduced from 947 to 15 due to restriction <15>
> 
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.000    0.000   10.889   10.889
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
>         1    0.000    0.000   10.127   10.127
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
>         1    0.000    0.000    9.047    9.047
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
>         1    4.523    4.523    8.653    8.653 {gtk._gtk.main}
>       244    0.030    0.000    3.011    0.012
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1642(on_task_modified)
>       244    0.027    0.000    2.595    0.011
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:53(update_tags_for_task)
>      4032    0.263    0.000    2.457    0.001 {method 'row_changed' of
> 'gtk.TreeModel' objects}
>      7208    0.394    0.000    1.734    0.000 {method 'get_value' of
> 'gtk.TreeModel' objects}
>      7495    0.060    0.000    1.380    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:82(on_get_value)
>       732    0.023    0.000    1.347    0.002
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:788(tag_visible_func)
>       879    0.297    0.000    1.295    0.001
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/tagstore.py:305(get_tasks_nbr)
>     12660    0.107    0.000    1.140    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/requester.py:135(get_task)
>         1    0.000    0.000    1.074    1.074
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
>     12660    0.231    0.000    1.034    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:51(get_task)
>      2709    0.175    0.000    0.825    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:858(tag_sort_func)
> 
> As you can see, a lot of time is spent in update_tags_for_task. If you
> comment this out, get_value and on_task_modified don't even show up in
> the profiling log:
> 
>    Ordered by: cumulative time
>    List reduced from 944 to 15 due to restriction <15>
> 
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    0.000    0.000    9.428    9.428
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
>         1    0.000    0.000    8.639    8.639
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
>         1    0.000    0.000    7.443    7.443
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
>         1    5.892    5.892    7.012    7.012 {gtk._gtk.main}
>         1    0.001    0.001    1.173    1.173
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
>         1    0.000    0.000    0.776    0.776
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine)
>         1    0.002    0.002    0.775    0.775
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins)
>        12    0.002    0.000    0.587    0.049
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__)
>        12    0.001    0.000    0.544    0.045
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module)
>        12    0.003    0.000    0.514    0.043 {imp.load_module}
>        29    0.003    0.000    0.442    0.015
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml)
>         2    0.000    0.000    0.430    0.215
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore)
>        10    0.356    0.036    0.395    0.040 {method 'show' of
> 'gtk.Widget' objects}
>       975    0.064    0.000    0.383    0.000
>
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tasktree.py:94(on_get_value)
>        29    0.001    0.000    0.334    0.012
> /usr/lib/python2.6/xml/dom/minidom.py:47(toprettyxml)
> 
> It's probably not the task of the browser to refresh each tag related
> to a task. Since Tags have references to their tasks, they are
> modified as well when a Task is edited. So we should probably let the
> tag emit a signal to warn everyone they changed (much like the Task
> are doing now). Let the objets which care about it to register
> themselve to this signal through the requester.
> 
> That would imply having a "on_tag_modified" in the browser that would
> trigger some job to perform on the browser (probably not much since,
> if I understood well, most of the filtering work is now done outside
> the browser)
> 
>> If you want to contribute, there's a huge cleaning to do in the
>> browser.py
>> ! A lot of cruft should be removed (and I don't understand everything
>> there).
> 
> I agree with what you say, but it would be more efficient if you could
> be more specific and point out some of these cruft.
> 
>>
>> What does not work :
>> - tag filtering. They are implemented.  We just need to do
>> "self.req.apply_filter(tagname)" when a particular tag is selected but
I
>> don't know where it is in browser.py and I'm a bit tired. (any help is
>> welcome here)
>> - drag-n-drop : code is commented out in tasktree.py. Maybe it will
work
>> without any trouble.
>> - multiparent (not tested yet)
>> - tag count in tagtree
>>
>> What is broken :
>> - plugins and plugin_api
>> - dbus interface
>>
>>
>> I was expecting better performances but FilteredTree appeared to be a
lot
>> more complex than expected. So now, it's time to optimize it a bit and
>> I'm
>> wondering where is the bottleneck.
> 
> Like I wrote above, it seems to be situated around tags update
> mechanisms and TagTreeModel updates.
> 
>>
>> Thanks for listenning,
>>
>>
>> Lionel
>>
>> _______________________________________________
>> Mailing list: https://launchpad.net/~gtg-contributors
>> Post to     : gtg-contributors@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~gtg-contributors
>> More help   : https://help.launchpad.net/ListHelp
>>