gtg-user team mailing list archive

[Jonathan Lange <jml@xxxxxxxxx>]

On Wed, Jan 6, 2010 at 10:02 PM, Lionel Dricot <ploum@xxxxxxxxx> wrote:
> I'm sorry for being really late and I wish you a very happy/funny/free
> as in speech and as in beer 2010.
>

I'm later still in my follow up!

So far, 2010 has been full of lots of free speech and much beer of
varying cost. :)

>
> Le jeudi 17 décembre 2009 à 18:12 +1100, Jonathan Lange a écrit :
>> On Thu, Dec 17, 2009 at 9:56 AM, Lionel Dricot <ploum@xxxxxxxxx> wrote:
>> > The following describes my vision of backends and storage in GTG. This
>> > is what I intended to do for 0.2. If you agree, it will be done for 0.3!
>> >
>>
>> Thanks for sharing the vision.
>>
>> I don't want to stop you from doing this work, but if you want to have
>> the best outcome, then it's worth being clear on exactly *why* you
>> want to do this, and on what the result will look like to the user.
>> Because I believe this, I'm going to ask quite a few questions.
>
> That's fair enough and exactly why I was writing this and not blindly
> implementing/comiting. Thanks for your reply.
>

My pleasure.

>
>>
>> > >From the very first lines of GTG, we knew that storing your tasks would
>> > be different for everybody. That's why we designed our backend
>> > architecture and we designed GTG to work with multiple backends at the
>> > same time.
>> >
>>
>> Why would storing tasks be different for everyone? I don't use Tomboy
>> any more, but I used it for years without caring how it stored my
>> notes. Why would I care about how GTG stores my tasks?
>
>
> Because, in fact, that's one of the biggest tomboy complain. The most
> requested feature ever for GTG (and perhaps for tomboy) is the online
> storage.
>
> But some want Ubuntu One, other Weddav server or a custom webservice
> like RTM.
>
> So in fact, people care about it. It's important to them and they want
> to back it up.
>
> Tomboy neglected that fact and only recently added a hack to have sync
> with Ubuntu One. It simply doesn't work : you have to manually press a
> button to sync, you have conflict, no error management.
>
> I do want GTG to have a seamless online experience but I want to keep it
> offline for those who prefer it that way.
>

That's an excellent goal. But I wonder whether it's better to provide
a generic system, or whether we should just do Ubuntu One (for
example) extremely well.

>
>>
>> > Did you know ? GTG already supports multiple backends since 0.1 ! If you
>> > manually add a second backend in the backends.xml, it will work like
>> > expected.
>> >
>>
>> I did know that. It makes me wonder, is there an explanation of the
>> layers of the GTG architecture?
>
> Yes. The backends layer was done to allow easy multi-backends.
>
> The datastore/requester layer was done to isolate as much as possible
> GTG core from the GUI to allow other GUI. It might be a Qt gui for KDE
> guys but it could be a webinterface.
>
> I admit : some loose bits are purely historical.
>

Personally, I'd love to do some hacking to make this a bit tighter,
and to change things so that layers only know about the layers *below*
them. I've held off because it seems like the kind of thing that would
best be done in close collaboration with you & Bertrand.


>>
>> > Use the already existant Luke! Do you see? Indeed, I'm looking at
>> > @tags !
>> >
>> > So, will it work ?
>> >
>> > I imagine 4 types of backend you can have : read-write, read-only,
>> > import, export.
>> >
>>
>> "Backend" is a pretty generic term. I know it has a particular use
>> within the GTG codebase, but I think that the discussion would be
>> clearer if we could find something better to use.
>
> Admittedly, this is becoming a GTGesque jargon ;-)
>
>>
>> > * Read-write is the traditionnal backend you already know. It will also
>> > be used for the couchdb backend and, in fact, most backends.
>>
>> Is there anything already in the code or documentation that specifies
>> what you must do to implement one of these? I think that writing such
>> a thing would be an excellent first step.
>>
>> Looking at GTG/backends/localfile.py, it seems that there's a bunch of
>> top-level functions that must be provided by a module, and a Backend
>> class.
>>
>> The purpose of most of the top-level methods is clear, but
>> get_parameters() seems pretty vague. It's not clear why some things
>> are top-level functions and other things are methods.
>
> Sorry for not explaining that. Top-level functions should be used by the
> GUI to display a list of available backends and build the preferences
> window for that backend.
>
> They are top-level because you want to call them without instantiating
> the backend object by itself.
>
> Methods, on the opposite, will be used only in a particular instance of
> that backend.
>
> Do you see my point ?
>

Yeah, I've always seen that point.

Thing is, the current system makes the various backend modules
singleton objects with a loosely-defined type of "backend format" or
"type of backend". One could also make an object like this:

class LocalFileBackend(object):
    def get_name(self):
        return "Local file"
    # etc.

    def make_backend(self, parameters, firstrunxml=None):
        # Probably would have more code, but with the current implementation...
        return Backend(parameters, firstrunxml=firstrunxml)

You'd then probably want to do something like:
  register_backend(LocalFileBackend)

This approach is easier to test, makes adding and removing backends
explicit (which is better than implicit).

>
>>
>> > * Read-only displays a list of tasks that you have to do but you should
>> > trigger something external to close them. It might be useful for
>> > backends like bugzilla (at least as a first step) or for centralized
>> > backend where your boss have to acknowledge that you finished a task.
>>
>> AIUI, the _only_ thing this gives you that import backends don't is
>> that it stops you from editing the tasks.
>>
>> In that case, rather than having multiple types of backends, why not
>> have a property on tasks that says whether or not that task is
>> read-only?
>
> Not at all. If you have a read-only backend, they can still be modified
> elsewhere. When you use an "import" backend, you create new tasks in
> your existing default backend.
>
> Do you see the difference ?
>

I'm sorry, but I don't. Are you saying that "import" means "import a
bunch of things *once*" and read-only means "keep importing a bunch of
things"?

>>
>> > * Import backends will simply take tasks from the backend, just as the
>> > user entered them manually. A typical example might me a Twitter
>> > backend. It would retrieve all of your tweets with the tag #todo and
>> > create a new task with them.
>>
>> I don't think it's helpful to think of this as a "backend". It's
>> actually two things:
>>   - a process for putting things into a backend
>>   - configuration to say how often this should take place
>
> I don't see how it is simpler than considering it as a backend.

Because knowledge of how tasks are stored in, say, Launchpad is a
fundamentally different kind of thing from "pull new tasks four times
every day". Who is to say I want my tasks from Launchpad to keep
importing?



>> > Each backend will define itself with a set of options (a bit like
>> > plugins). In order to not copy/paste too much, we might have some
>> > predefined backends that are instantiated by others. For example, we
>> > will have a local-text-file backend for which the user can choose the
>> > location of his datas. The default backend will only be an instantiation
>> > of that backend but with a fixed data path (value=XDG_HOME_DATA).
>> >
>>
>> That's a great idea. Another way of thinking about it is that you want
>> an object to represent a format (what we call a backend), and an
>> object to represent a document of that format (what we call a
>> tasksource). It should be possible to create a document of a
>> particular format from the format object.
>>
>> The critical thing is that there has to be a clearly defined interface
>> that any valid TaskSource must implement. Ideally, we'd have unit
>> tests for this interface, and run these tests against all known types
>> of tasksource.
>
> Don't you mean backend ? Tasksources are instance of Backends.
>

Uhhh, maybe. I said "types of TaskSource", which I think means
"Backends". Maybe we need better words :)


>> > So, what's my plan to do this?
>> >
>> > Before everything (step 0), I want to change and improve the backends
>> > API. For example, the get_all will no longer returns every single task
>> > from the dawn of humanity but will instead retrieve opened tasks and
>> > tasks that were closed less that X days ago where X is a parameter,
>> > common to all backends, that the user can change. This will solve :
>> > https://bugs.edge.launchpad.net/gtg/+bug/312700
>> > https://bugs.edge.launchpad.net/gtg/+bug/495758
>> >
>>
>> Better yet, don't have a get_all method: have something that supports
>> a streaming approach.
>
> We will refactorize that to use callbacks so the get_all will, in fact,
> ask the backend to add each task himself.

OK. I guess that will work. I mostly meant you should make get_all
return an iterator or something similar.

>> > Second step is ensure that tasks could support multiple IDs from
>> > differents tasksources.
>> >
>>
>> Why?
>
> Because a task could be part of multiple tasksources.
>

I don't mean to be childish, but...

Why?

>>
>> > Third step : add support to assign a specific backends based on tags.
>> > Yes, it means that tasks migth be first added to the default backend
>> > then removed from it once a tag is added.
>> >
>>
>> Actually, no. The next step is having a user interface for configuring
>> a backend at all.
>>
>> Then the next step is having support for _multiple_ tasksources with
>> one default write tasksources. This includes things like being able to
>> look at a task and see which tasksource it is from. It will also
>> include things like disabling features based on backend.
>>
>> *Then* you get to having support based on assigning a tasksource to a tag.
>
> Agree but I hate Glade so much that I always put it as the last bit :-D
>

Maybe you should try out Balsamiq Mockups. It's a pretty neat way of
mocking up an interface, and might help with this.

>> > Fifth : put a GUI above all of that so the user can choose and configure
>> > his own tasksources.
>> >
>>
>> As mentioned before, I think you should do this sooner. None of the
>> earlier steps have any user benefit until you can do this.
>
> Agree but, hey, it's glade man ! ;-)
>

Yeah, but work that doesn't help users is waste.

>>
>> >
>> > This is the vision of a multi-backends tasks system I developed for more
>> > than one year. This is the last big stuff I want to add to GTG before a
>> > 1.0 release. Of course, nothing is written in stone. I'm open to critics
>> > and proposition (mostly from coding people as this is a quite technical
>> > proposition). Also remember that GTG is a community project and that my
>> > vision will only be accepted if I can convince the majority of
>> > developers that this is a good idea.
>>
>> Thank you for putting your vision on the line.
>>
>> I realize that I've been a bit critical. This is partly because I have
>> a bias against big infrastructural changes, but more because I care
>> about GTG and want it to keep doing well.
>
>
> I appreciate your critics. As you can see, I changed my mind for some
> stuffs. I hope I answered your question.
>
> Your contribution to GTG is highly valuable and I'm glad to have you in
> our little community. Don't hesitate to send critical mails like that,
> it shapes things for a better future.
>
>
> Pfew, I answered it !!!
>

And so gracefully too! :)

> Thanks for that,
>

Thank you! I look forward to seeing this backend work actually happen
& start helping end users.

jml