gtg-contributors team mailing list archive

[Bertrand Rousseau <bertrand.rousseau@xxxxxxxxx>]

On Wed, Feb 24, 2010 at 5:57 PM, Lionel Dricot <ploum@xxxxxxxxx> wrote:
>
> I've very good news for myself : I see the light.
>
> The FilteredTree is now working nearly perfectly. It is rock-solid, even
> when loading 300 tasks at the same time. I also implemented the Closed
> Tasks pane with it. The FiltersBank is also working great.

Great news!

> Bad news : with a lot of tasks, the start time is really really really
> slow. (like 10 times the current trunk launch time). On the other hand, it
> looks like other performance issues are mostly solved (like renaming a
> subtask in a task).

Task renaming is maybe due to the fact that you disabled the refresh
methods from the browser. Originally, it can from the fact that
browser refresh is called at each letter you type by the editor. It
should be worth looking at this too or it will probably show again
sometime.

> So now, I need *you*. I need testing and profiling.
>
> Do : bzr branch lp:~gtg/gtg/gtg-refactor
> and launch it first with debugging data then with actual data. Do some
> profiling ( http://live.gnome.org/gtg/development ) and send me the
> results.

Here's mine (on my netbook, so slownesses are sensible here):

   Ordered by: cumulative time
   List reduced from 947 to 15 due to restriction <15>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   10.889   10.889
/home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
        1    0.000    0.000   10.127   10.127
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
        1    0.000    0.000    9.047    9.047
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
        1    4.523    4.523    8.653    8.653 {gtk._gtk.main}
      244    0.030    0.000    3.011    0.012
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1642(on_task_modified)
      244    0.027    0.000    2.595    0.011
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:53(update_tags_for_task)
     4032    0.263    0.000    2.457    0.001 {method 'row_changed' of
'gtk.TreeModel' objects}
     7208    0.394    0.000    1.734    0.000 {method 'get_value' of
'gtk.TreeModel' objects}
     7495    0.060    0.000    1.380    0.000
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:82(on_get_value)
      732    0.023    0.000    1.347    0.002
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:788(tag_visible_func)
      879    0.297    0.000    1.295    0.001
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/tagstore.py:305(get_tasks_nbr)
    12660    0.107    0.000    1.140    0.000
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/requester.py:135(get_task)
        1    0.000    0.000    1.074    1.074
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
    12660    0.231    0.000    1.034    0.000
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:51(get_task)
     2709    0.175    0.000    0.825    0.000
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:858(tag_sort_func)

As you can see, a lot of time is spent in update_tags_for_task. If you
comment this out, get_value and on_task_modified don't even show up in
the profiling log:

   Ordered by: cumulative time
   List reduced from 944 to 15 due to restriction <15>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    9.428    9.428
/home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
        1    0.000    0.000    8.639    8.639
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
        1    0.000    0.000    7.443    7.443
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
        1    5.892    5.892    7.012    7.012 {gtk._gtk.main}
        1    0.001    0.001    1.173    1.173
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
        1    0.000    0.000    0.776    0.776
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine)
        1    0.002    0.002    0.775    0.775
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins)
       12    0.002    0.000    0.587    0.049
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__)
       12    0.001    0.000    0.544    0.045
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module)
       12    0.003    0.000    0.514    0.043 {imp.load_module}
       29    0.003    0.000    0.442    0.015
/home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml)
        2    0.000    0.000    0.430    0.215
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore)
       10    0.356    0.036    0.395    0.040 {method 'show' of
'gtk.Widget' objects}
      975    0.064    0.000    0.383    0.000
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tasktree.py:94(on_get_value)
       29    0.001    0.000    0.334    0.012
/usr/lib/python2.6/xml/dom/minidom.py:47(toprettyxml)

It's probably not the task of the browser to refresh each tag related
to a task. Since Tags have references to their tasks, they are
modified as well when a Task is edited. So we should probably let the
tag emit a signal to warn everyone they changed (much like the Task
are doing now). Let the objets which care about it to register
themselve to this signal through the requester.

That would imply having a "on_tag_modified" in the browser that would
trigger some job to perform on the browser (probably not much since,
if I understood well, most of the filtering work is now done outside
the browser)

> If you want to contribute, there's a huge cleaning to do in the browser.py
> ! A lot of cruft should be removed (and I don't understand everything
> there).

I agree with what you say, but it would be more efficient if you could
be more specific and point out some of these cruft.

>
> What does not work :
> - tag filtering. They are implemented.  We just need to do
> "self.req.apply_filter(tagname)" when a particular tag is selected but I
> don't know where it is in browser.py and I'm a bit tired. (any help is
> welcome here)
> - drag-n-drop : code is commented out in tasktree.py. Maybe it will work
> without any trouble.
> - multiparent (not tested yet)
> - tag count in tagtree
>
> What is broken :
> - plugins and plugin_api
> - dbus interface
>
>
> I was expecting better performances but FilteredTree appeared to be a lot
> more complex than expected. So now, it's time to optimize it a bit and I'm
> wondering where is the bottleneck.

Like I wrote above, it seems to be situated around tags update
mechanisms and TagTreeModel updates.

>
> Thanks for listenning,
>
>
> Lionel
>
> _______________________________________________
> Mailing list: https://launchpad.net/~gtg-contributors
> Post to     : gtg-contributors@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~gtg-contributors
> More help   : https://help.launchpad.net/ListHelp
>



-- 
Bertrand Rousseau