multi-touch-dev team mailing list archive

[Peter Hutterer <peter.hutterer@xxxxxxxxx>]

On Wed, Oct 13, 2010 at 08:43:42PM +0200, Henrik Rydberg wrote:
> On 10/13/2010 06:03 PM, Chase Douglas wrote:
> [...]
> 
> >> This is under the assumption that we don't add regions to geis, or any other way
> >> of obtaining windowing information from the toolkits.
> > 
> > Correct. I'll try to answer the unstated question of whether we should
> > do this with a single geis daemon instance.
> > 
> > Imagine you've got a window with many widgets. One of them is a
> > scrollable region, so it subscribes to gestures and provides the region
> > in screen coordinates.
> > 
> > Now the window is moved on screen. Should the toolkit have to update
> > geis with the new location? If so, it sounds racy. The WM could infer
> > the new position. However, what happens when the window is resized? The
> > new area of the scrollable widget is toolkit and application specific,
> > so the toolkit will need to inform geis of the new region. Again, I see
> > things getting racy during a resize operation. It also adds a bunch of
> > round trip messages and potential for UI delay. There may also be
> > contention for the geis daemon between clients.
> 
> 
> Um... a lot of "could be racy" and "potential delay" in this paragraph, without
> actually backing it up. ;-)

if the window must be manually updated and the events to geis
doesn't have the same delivery path as however you update the hierarch, you
_will_ have race conditions. the only way to get around that is if whatever
hands events to geis also hands the window update information before the new
events come in.

> > There's a few context questions we need to ask first (I'm trying to get
> > at what your asking, maybe I'm going off in the wrong direction though):
> > 
> > 1. Is a three finger pinch a system-wide gesture? I'll assume yes. Are
> > two finger gestures system-wide gestures? I'll assume no.
> > 
> > 2. Should a two-to-three finger movement that is not immediate, by which
> > I mean this isn't a three finger movement from the start where the first
> > couple of device touch frames caught only two fingers, trigger a three
> > finger gesture? I don't believe we've explicitly answered this anywhere
> > yet, but I think not. A three finger gesture must be initiated with
> > three fingers within a short period of time on the order of 10s of
> > milliseconds (I'm leaving aside the case of gesture continuation).
> 
> 
> I tend to agree, although that is all about policy. I was intending this example
> to trigger a global gesture, since it makes the example more difficult.
> 
> > Given these assumptions, I think the expected result is that the
> > scrollable region continues to scroll according to the two fingers over
> > it, and the single finger in the other window does whatever it should in
> > that window.
> 
> 
> Right, that is the easy path. So what about the one which triggers a global gesture?
> 
> 
> > From the plumbing perspective I see the following sequence of events:
> > 
> > 1. Two fingers touch down in scrollable region
> > 2. MT data passed to WM through XI 2.1 passive grab
> > 3. MT data also passed to app toolkit, but with "not-for-you-yet" flag
> > 4. WM and toolkit pass data to their own instances of geis/grail
> > 5. WM grail times out waiting for three fingers
> > 6. WM gets MT data back, realizes there's no gesture, "replays" touches
> > 7. In parallel, app toolkit gets two finger drag gesture from geis/grail
> > 8. App toolkit gets event saying MT events "are for you now"
> > 9. App toolkit starts scrolling
> > 10. Third finger goes down in another window
> > 11. Steps 2-6 and 8 repeated for this touch
> > 12. App toolkit for this other window handles touch as appropriate
> > 
> 
> 
> Thank you for this thorough example. I do think it is quite complex, and fear
> there will be lag creeping in at point 2, 3, 4 and 8 in the chain.

yep, there will be. the trick is to make this lag appear irrelevant to the
user. there's a lag every time you click into a window because the WM gets
the event first and then replays it on the actual window. it's usually not
noticable to the user though, even when you know the lag is there.

> > Note that after the WM replays any touches, no more events generated by
> > those touches are sent to it. It will only get events from new touches
> > to the screen.
> 
> I am not entirely sure how 6 works, and what "gets MT data back" means.

gets data back from grail, saying gesture is not a WM gesture (my guess).

> > Also, during the "not-for-you-yet" phase, applications can provide UI
> > feedback or do anything else with the data as long as they appropriately
> > handle both cases where the touches end up replayed or consumed by the
> > grabbing client.
> 
> 
> I am afraid the not-for-you events would normally appear during a phase in the
> gesture when nothing at all should happen, like the onset of a rotation. The
> second phase, when the rotation is detected but yet to be committed by the user,
> will there be a not-for-you message to the app? And in that case, what would
> happen with the ability to notify the user of the ongoing rotation?

I think there may be a misunderstanding of the "not for you" flag.
There are two reasons why this came up. one is that doing so the server
doesn't have to cache events indefinitely if there is a sync grab going on.
It can flush the events out but mark them as "do not use" (this was the main
motivation).
The other one is that gesture detection can be performed in parallel in
multiple clients. The only benefit you get is deserialization.
given the replay case, the event sequence changes from:

[grab] [events to client1] [gesture detection] [replay request] [events to
client2] [gesture detection] [action]

to
[grab] [ev to client1] [gest det] [replay req] 
       [ev to client2] [gest det]             ["use it" event] [action]

(graphic best viewed in monospace or Internet Explorer 2.0 at 640x234 :)

this parallelisation is the only real effect. the "use it" events simply
shorten the delivery when a set of events is given the OK for a 
client - that client already has the data by then.

note that we haven't actually decided on doing this, because I predict that
the details of getting this semantically sound in the server will be quite
tricky.

Cheers,
  Peter

> > I hope that covers the use case and I'm not forgetting anything :).
> > Thanks for asking about this scenario, I've been meaning to post a
> > detailed event sequence but just haven't gotten around to it yet.
> 
> 
> Thanks a lot, Chase, it certainly helps to get down to this level of detail.