multi-touch-dev team mailing list archive

[Henrik Rydberg <rydberg@xxxxxxxxxxx>]

On 10/14/2010 09:39 PM, Chase Douglas wrote:

> On Wed, 2010-10-13 at 20:43 +0200, Henrik Rydberg wrote:
>> On 10/13/2010 06:03 PM, Chase Douglas wrote:
>>>> Good point. Would you mind going through, step-by-step, the case where we start
>>>> with two fingers inside a "window", scroll a bit, then add another finger in a
>>>> different window, and perform a three-finger pinch?
>>>
>>> There's a few context questions we need to ask first (I'm trying to get
>>> at what your asking, maybe I'm going off in the wrong direction though):
>>>
>>> 1. Is a three finger pinch a system-wide gesture? I'll assume yes. Are
>>> two finger gestures system-wide gestures? I'll assume no.
>>>
>>> 2. Should a two-to-three finger movement that is not immediate, by which
>>> I mean this isn't a three finger movement from the start where the first
>>> couple of device touch frames caught only two fingers, trigger a three
>>> finger gesture? I don't believe we've explicitly answered this anywhere
>>> yet, but I think not. A three finger gesture must be initiated with
>>> three fingers within a short period of time on the order of 10s of
>>> milliseconds (I'm leaving aside the case of gesture continuation).
>>
>>
>> I tend to agree, although that is all about policy. I was intending this example
>> to trigger a global gesture, since it makes the example more difficult.
>>
>>> Given these assumptions, I think the expected result is that the
>>> scrollable region continues to scroll according to the two fingers over
>>> it, and the single finger in the other window does whatever it should in
>>> that window.
>>
>>
>> Right, that is the easy path. So what about the one which triggers a global gesture?
> 
> I'm not sure we've defined a mechanism that would work in that case.
> What you would need is the ability to peek at all touch events
> individually and pick and choose which ones you want to keep and which
> you want to replay. The reason this gets dicey is what does the client
> do? Imagine the following:
> 
> 1. Touch begins
> 2. WM replays touch events one at a time
> 3. Application gets touch begin, touch motion events
> 4. WM consumes some events
> 5. WM replays more events
> 6. WM consumes events
> 7. Touch ends
> 
> Issues:
> 
> * Does application get touch begin events when touch physically begins
> if WM starts consuming all events? Or does it only receive a begin event
> once the first replayed event is sent?
> * Is application notified when WM starts consuming events again?
> * What happens when events are replayed in step 5? It may seem that the
> touch moved from one are of the screen to another instantaneously when
> in reality the touches in between were consumed by the WM.
> * If the WM consumes the touch end, does the application then receive
> the touch end?
> 
> I think this use case is opposite to good UI design, and I think the
> plumbing to make it possible opens up a big can of worms. X doesn't
> support any replay mechanism like this for even legacy pointer events.
> We seem to all agree that the original example bore the correct outcome,
> so I'm hesitant to continue down this path until we find a scenario that
> would require it.
> 
> There's also nothing that prevents us from adding support for this in
> the future if we find we need it.


Thanks for showing the implications, Chase. I think it illustrates well that MT
and Gestures cannot be separated.

If we think of MT events as separate single-finger drags, MT becomes a subset of
gestures. In grail, a N-finger gesture overrides all other gestures that can be
performed with M < N fingers. This includes one-finger drags. Going back to your
example, the natural interpretation is that when the three-finger gesture
starts, the two single-finger drags end. Perhaps if XI2.1 had the ability to
chop a touch up into pieces during replay, the problem could be solved.

This thread, as well as talking some more with Peter, makes me think we are
pretty close. The things I see we need to do is:

1. Add the ability to limit finger grouping to areas in grail.

The separation into finger groups is already there, but it is only based on the
incoming data. Adding the ability to split the fingers into groups based on
domains could be achieved by an api callback.

The visible simplifications during natty would be more expressive power to the
application. In particular, the ability to have a dedicated scroll area. It
would also enable the possibility to perform all event propagation from within
the same process, although we would not push this scenario at all.

2. Add the ability to split touches in XI2.1

The passive grab mechanism needs to be able to gracefully handle consumption of
touches, such that touch-up and touch-down events are inserted appropriately if
the consumer of a touch changes.

3. Skip the tentative events in XI2.1

>From Peter's explanation, it seems the original reason for this was to not run
out of buffer space. Since the semantics is still unclear (just follow this
thread for an example), it is probably best pushed to future expansions.

So, Gentlemen, what do you think of this?

Cheers,
Henrik