gtg-contributors team mailing list archive
-
gtg-contributors team
-
Mailing list archive
-
Message #00039
Re: Refactorisation progress : It works (but it's slow)
On Thu, Feb 25, 2010 at 10:14 AM, Lionel Dricot <ploum@xxxxxxxxx> wrote:
>
> Only for performances, could you please take the latest revision (658) of
> my refactoring branch and try it with a lot of tasks and confirm me that
> I'm not dreaming ?
It seems quite fast, yes, but it's not using update_tags_for_task either:
109098 function calls (105962 primitive calls) in 3.730 CPU seconds
Ordered by: cumulative time
List reduced from 1042 to 15 due to restriction <15>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 3.730 3.730
/home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
1 0.000 0.000 3.567 3.567
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
1 0.000 0.000 3.428 3.428
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
1 3.313 3.313 3.331 3.331 {gtk._gtk.main}
1 0.000 0.000 0.139 0.139
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
10 0.093 0.009 0.096 0.010 {method 'show' of
'gtk.Widget' objects}
1 0.000 0.000 0.069 0.069
/home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine)
1 0.000 0.000 0.068 0.068
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins)
46 0.001 0.000 0.068 0.001
/home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml)
12 0.000 0.000 0.065 0.005
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__)
12 0.000 0.000 0.062 0.005
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module)
2 0.000 0.000 0.061 0.031
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore)
12 0.001 0.000 0.060 0.005 {imp.load_module}
1 0.000 0.000 0.049 0.049
/home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:66(check_instance)
2 0.000 0.000 0.045 0.023
/home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:203(quit)
> I just can't believe what I'm seeing but that computer is fast.
>
>
> Lionel
>
> PS: tags display is still broken.
>
>
>
> On Thu, 25 Feb 2010 01:27:16 +0100, Bertrand Rousseau
> <bertrand.rousseau@xxxxxxxxx> wrote:
>> On Wed, Feb 24, 2010 at 5:57 PM, Lionel Dricot <ploum@xxxxxxxxx> wrote:
>>>
>>> I've very good news for myself : I see the light.
>>>
>>> The FilteredTree is now working nearly perfectly. It is rock-solid,
> even
>>> when loading 300 tasks at the same time. I also implemented the Closed
>>> Tasks pane with it. The FiltersBank is also working great.
>>
>> Great news!
>>
>>> Bad news : with a lot of tasks, the start time is really really really
>>> slow. (like 10 times the current trunk launch time). On the other hand,
>>> it
>>> looks like other performance issues are mostly solved (like renaming a
>>> subtask in a task).
>>
>> Task renaming is maybe due to the fact that you disabled the refresh
>> methods from the browser. Originally, it can from the fact that
>> browser refresh is called at each letter you type by the editor. It
>> should be worth looking at this too or it will probably show again
>> sometime.
>>
>>> So now, I need *you*. I need testing and profiling.
>>>
>>> Do : bzr branch lp:~gtg/gtg/gtg-refactor
>>> and launch it first with debugging data then with actual data. Do some
>>> profiling ( http://live.gnome.org/gtg/development ) and send me the
>>> results.
>>
>> Here's mine (on my netbook, so slownesses are sensible here):
>>
>> Ordered by: cumulative time
>> List reduced from 947 to 15 due to restriction <15>
>>
>> ncalls tottime percall cumtime percall filename:lineno(function)
>> 1 0.000 0.000 10.889 10.889
>> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
>> 1 0.000 0.000 10.127 10.127
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
>> 1 0.000 0.000 9.047 9.047
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
>> 1 4.523 4.523 8.653 8.653 {gtk._gtk.main}
>> 244 0.030 0.000 3.011 0.012
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1642(on_task_modified)
>> 244 0.027 0.000 2.595 0.011
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:53(update_tags_for_task)
>> 4032 0.263 0.000 2.457 0.001 {method 'row_changed' of
>> 'gtk.TreeModel' objects}
>> 7208 0.394 0.000 1.734 0.000 {method 'get_value' of
>> 'gtk.TreeModel' objects}
>> 7495 0.060 0.000 1.380 0.000
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tagtree.py:82(on_get_value)
>> 732 0.023 0.000 1.347 0.002
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:788(tag_visible_func)
>> 879 0.297 0.000 1.295 0.001
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/tagstore.py:305(get_tasks_nbr)
>> 12660 0.107 0.000 1.140 0.000
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/requester.py:135(get_task)
>> 1 0.000 0.000 1.074 1.074
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
>> 12660 0.231 0.000 1.034 0.000
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/datastore.py:51(get_task)
>> 2709 0.175 0.000 0.825 0.000
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:858(tag_sort_func)
>>
>> As you can see, a lot of time is spent in update_tags_for_task. If you
>> comment this out, get_value and on_task_modified don't even show up in
>> the profiling log:
>>
>> Ordered by: cumulative time
>> List reduced from 944 to 15 due to restriction <15>
>>
>> ncalls tottime percall cumtime percall filename:lineno(function)
>> 1 0.000 0.000 9.428 9.428
>> /home/rousseau/workspace/gtg/gtg-refactor/GTG/gtg.py:91(main)
>> 1 0.000 0.000 8.639 8.639
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/manager.py:49(show_browser)
>> 1 0.000 0.000 7.443 7.443
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:1795(main)
>> 1 5.892 5.892 7.012 7.012 {gtk._gtk.main}
>> 1 0.001 0.001 1.173 1.173
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:83(__init__)
>> 1 0.000 0.000 0.776 0.776
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/browser.py:525(_init_plugin_engine)
>> 1 0.002 0.002 0.775 0.775
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:147(load_plugins)
>> 12 0.002 0.000 0.587 0.049
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:40(__init__)
>> 12 0.001 0.000 0.544 0.045
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/plugins/engine.py:108(_load_module)
>> 12 0.003 0.000 0.514 0.043 {imp.load_module}
>> 29 0.003 0.000 0.442 0.015
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/tools/cleanxml.py:98(savexml)
>> 2 0.000 0.000 0.430 0.215
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/core/__init__.py:160(save_datastore)
>> 10 0.356 0.036 0.395 0.040 {method 'show' of
>> 'gtk.Widget' objects}
>> 975 0.064 0.000 0.383 0.000
>>
> /home/rousseau/workspace/gtg/gtg-refactor/GTG/taskbrowser/tasktree.py:94(on_get_value)
>> 29 0.001 0.000 0.334 0.012
>> /usr/lib/python2.6/xml/dom/minidom.py:47(toprettyxml)
>>
>> It's probably not the task of the browser to refresh each tag related
>> to a task. Since Tags have references to their tasks, they are
>> modified as well when a Task is edited. So we should probably let the
>> tag emit a signal to warn everyone they changed (much like the Task
>> are doing now). Let the objets which care about it to register
>> themselve to this signal through the requester.
>>
>> That would imply having a "on_tag_modified" in the browser that would
>> trigger some job to perform on the browser (probably not much since,
>> if I understood well, most of the filtering work is now done outside
>> the browser)
>>
>>> If you want to contribute, there's a huge cleaning to do in the
>>> browser.py
>>> ! A lot of cruft should be removed (and I don't understand everything
>>> there).
>>
>> I agree with what you say, but it would be more efficient if you could
>> be more specific and point out some of these cruft.
>>
>>>
>>> What does not work :
>>> - tag filtering. They are implemented. We just need to do
>>> "self.req.apply_filter(tagname)" when a particular tag is selected but
> I
>>> don't know where it is in browser.py and I'm a bit tired. (any help is
>>> welcome here)
>>> - drag-n-drop : code is commented out in tasktree.py. Maybe it will
> work
>>> without any trouble.
>>> - multiparent (not tested yet)
>>> - tag count in tagtree
>>>
>>> What is broken :
>>> - plugins and plugin_api
>>> - dbus interface
>>>
>>>
>>> I was expecting better performances but FilteredTree appeared to be a
> lot
>>> more complex than expected. So now, it's time to optimize it a bit and
>>> I'm
>>> wondering where is the bottleneck.
>>
>> Like I wrote above, it seems to be situated around tags update
>> mechanisms and TagTreeModel updates.
>>
>>>
>>> Thanks for listenning,
>>>
>>>
>>> Lionel
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~gtg-contributors
>>> Post to : gtg-contributors@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~gtg-contributors
>>> More help : https://help.launchpad.net/ListHelp
>>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~gtg-contributors
> Post to : gtg-contributors@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~gtg-contributors
> More help : https://help.launchpad.net/ListHelp
>
--
Bertrand Rousseau
Follow ups
References
