multi-touch-dev team mailing list archive

[Henrik Rydberg <rydberg@xxxxxxxxxxx>]

On 10/13/2010 06:03 PM, Chase Douglas wrote:
[...]

>> This is under the assumption that we don't add regions to geis, or any other way
>> of obtaining windowing information from the toolkits.
> 
> Correct. I'll try to answer the unstated question of whether we should
> do this with a single geis daemon instance.
> 
> Imagine you've got a window with many widgets. One of them is a
> scrollable region, so it subscribes to gestures and provides the region
> in screen coordinates.
> 
> Now the window is moved on screen. Should the toolkit have to update
> geis with the new location? If so, it sounds racy. The WM could infer
> the new position. However, what happens when the window is resized? The
> new area of the scrollable widget is toolkit and application specific,
> so the toolkit will need to inform geis of the new region. Again, I see
> things getting racy during a resize operation. It also adds a bunch of
> round trip messages and potential for UI delay. There may also be
> contention for the geis daemon between clients.


Um... a lot of "could be racy" and "potential delay" in this paragraph, without
actually backing it up. ;-)


> In short, I think it could be done this way, but I don't think it's
> optimal.

> 
>>>>> Also, in order to perform proper propagation between "windows", each
>>>>> application will need to query the window stack on every new input event
>>>>> to be sure it is allowed to handle the input.
>>>>
>>>>
>>>> Definitely not. The point is that the process doing propagation can have the
>>>> full picture because it is placed where all information is available.
>>>
>>> The X process space is the only place where window position information
>>> is available internally.
>>
>>
>> We went half-way towards extracting that information once already, so we know it
>> is a bit cumbersome, but possible.
> 
> It's possible, but I think it would be racy again. I also worry about
> interactions with X input grabbing (not just XI 2.1 MT grabbing, but
> even normal ST grabbing). It doesn't strike me as a better or more
> elegant solution.


What would become more elegant is the propagation solution once all window
information is available.


>>>> It is clear that the part still unsolved has to do with multi-user input. I
>>>> include the question whether a gesture should in fact be seen as a set of
>>>> individual fingers. Combinatorics, projections, greedy versus global... I think
>>>> we need a handle on this problem before deciding what direction to take.
>>>
>>> Certainly skepticism is necessary as we formulate a solution, so thanks
>>> for trying to highlight issues. In my mind I don't see any real issues
>>> though. Do you have any scenarios where the proposed mechanisms break
>>> down?
>>
>>
>> Good point. Would you mind going through, step-by-step, the case where we start
>> with two fingers inside a "window", scroll a bit, then add another finger in a
>> different window, and perform a three-finger pinch?
> 
> There's a few context questions we need to ask first (I'm trying to get
> at what your asking, maybe I'm going off in the wrong direction though):
> 
> 1. Is a three finger pinch a system-wide gesture? I'll assume yes. Are
> two finger gestures system-wide gestures? I'll assume no.
> 
> 2. Should a two-to-three finger movement that is not immediate, by which
> I mean this isn't a three finger movement from the start where the first
> couple of device touch frames caught only two fingers, trigger a three
> finger gesture? I don't believe we've explicitly answered this anywhere
> yet, but I think not. A three finger gesture must be initiated with
> three fingers within a short period of time on the order of 10s of
> milliseconds (I'm leaving aside the case of gesture continuation).


I tend to agree, although that is all about policy. I was intending this example
to trigger a global gesture, since it makes the example more difficult.

> Given these assumptions, I think the expected result is that the
> scrollable region continues to scroll according to the two fingers over
> it, and the single finger in the other window does whatever it should in
> that window.


Right, that is the easy path. So what about the one which triggers a global gesture?


> From the plumbing perspective I see the following sequence of events:
> 
> 1. Two fingers touch down in scrollable region
> 2. MT data passed to WM through XI 2.1 passive grab
> 3. MT data also passed to app toolkit, but with "not-for-you-yet" flag
> 4. WM and toolkit pass data to their own instances of geis/grail
> 5. WM grail times out waiting for three fingers
> 6. WM gets MT data back, realizes there's no gesture, "replays" touches
> 7. In parallel, app toolkit gets two finger drag gesture from geis/grail
> 8. App toolkit gets event saying MT events "are for you now"
> 9. App toolkit starts scrolling
> 10. Third finger goes down in another window
> 11. Steps 2-6 and 8 repeated for this touch
> 12. App toolkit for this other window handles touch as appropriate
> 


Thank you for this thorough example. I do think it is quite complex, and fear
there will be lag creeping in at point 2, 3, 4 and 8 in the chain.

> Note that after the WM replays any touches, no more events generated by
> those touches are sent to it. It will only get events from new touches
> to the screen.


I am not entirely sure how 6 works, and what "gets MT data back" means.

> Also, during the "not-for-you-yet" phase, applications can provide UI
> feedback or do anything else with the data as long as they appropriately
> handle both cases where the touches end up replayed or consumed by the
> grabbing client.


I am afraid the not-for-you events would normally appear during a phase in the
gesture when nothing at all should happen, like the onset of a rotation. The
second phase, when the rotation is detected but yet to be committed by the user,
will there be a not-for-you message to the app? And in that case, what would
happen with the ability to notify the user of the ongoing rotation?

> I hope that covers the use case and I'm not forgetting anything :).
> Thanks for asking about this scenario, I've been meaning to post a
> detailed event sequence but just haven't gotten around to it yet.


Thanks a lot, Chase, it certainly helps to get down to this level of detail.

Cheers,
Henrik