multi-touch-dev team mailing list archive

[Peter Hutterer <peter.hutterer@xxxxxxxxx>]

On Fri, Oct 15, 2010 at 05:23:43PM +0200, Henrik Rydberg wrote:
> On 10/14/2010 09:39 PM, Chase Douglas wrote:
> 
> > On Wed, 2010-10-13 at 20:43 +0200, Henrik Rydberg wrote:
> >> On 10/13/2010 06:03 PM, Chase Douglas wrote:
> >>>> Good point. Would you mind going through, step-by-step, the case where we start
> >>>> with two fingers inside a "window", scroll a bit, then add another finger in a
> >>>> different window, and perform a three-finger pinch?
> >>>
> >>> There's a few context questions we need to ask first (I'm trying to get
> >>> at what your asking, maybe I'm going off in the wrong direction though):
> >>>
> >>> 1. Is a three finger pinch a system-wide gesture? I'll assume yes. Are
> >>> two finger gestures system-wide gestures? I'll assume no.
> >>>
> >>> 2. Should a two-to-three finger movement that is not immediate, by which
> >>> I mean this isn't a three finger movement from the start where the first
> >>> couple of device touch frames caught only two fingers, trigger a three
> >>> finger gesture? I don't believe we've explicitly answered this anywhere
> >>> yet, but I think not. A three finger gesture must be initiated with
> >>> three fingers within a short period of time on the order of 10s of
> >>> milliseconds (I'm leaving aside the case of gesture continuation).
> >>
> >>
> >> I tend to agree, although that is all about policy. I was intending this example
> >> to trigger a global gesture, since it makes the example more difficult.
> >>
> >>> Given these assumptions, I think the expected result is that the
> >>> scrollable region continues to scroll according to the two fingers over
> >>> it, and the single finger in the other window does whatever it should in
> >>> that window.
> >>
> >>
> >> Right, that is the easy path. So what about the one which triggers a global gesture?
> > 
> > I'm not sure we've defined a mechanism that would work in that case.
> > What you would need is the ability to peek at all touch events
> > individually and pick and choose which ones you want to keep and which
> > you want to replay. The reason this gets dicey is what does the client
> > do? Imagine the following:
> > 
> > 1. Touch begins
> > 2. WM replays touch events one at a time
> > 3. Application gets touch begin, touch motion events
> > 4. WM consumes some events
> > 5. WM replays more events
> > 6. WM consumes events
> > 7. Touch ends
> > 
> > Issues:
> > 
> > * Does application get touch begin events when touch physically begins
> > if WM starts consuming all events? Or does it only receive a begin event
> > once the first replayed event is sent?
> > * Is application notified when WM starts consuming events again?
> > * What happens when events are replayed in step 5? It may seem that the
> > touch moved from one are of the screen to another instantaneously when
> > in reality the touches in between were consumed by the WM.
> > * If the WM consumes the touch end, does the application then receive
> > the touch end?
> > 
> > I think this use case is opposite to good UI design, and I think the
> > plumbing to make it possible opens up a big can of worms. X doesn't
> > support any replay mechanism like this for even legacy pointer events.
> > We seem to all agree that the original example bore the correct outcome,
> > so I'm hesitant to continue down this path until we find a scenario that
> > would require it.
> > 
> > There's also nothing that prevents us from adding support for this in
> > the future if we find we need it.
> 
> 
> Thanks for showing the implications, Chase. I think it illustrates well that MT
> and Gestures cannot be separated.
> 
> If we think of MT events as separate single-finger drags, MT becomes a subset of
> gestures. 

I disagree with this point. Unless you expand the gesture dictionary ad
infintum, you will always find some combination of single-finger drags that
do not combine into a gesture.

> In grail, a N-finger gesture overrides all other gestures that can be
> performed with M < N fingers. 

At which point do you decide that N fingers are an N-finger gesture and not
just a random bunch of touch events? This is quite important for my
understanding, because right now I feel we're not always talking about the
same thing.

> This includes one-finger drags. Going back to your
> example, the natural interpretation is that when the three-finger gesture
> starts, the two single-finger drags end. 

[citation needed]
I'm actually being serious here. "natural" is quite tricky to define and I'd
love to see the research that confirms this statement.

> Perhaps if XI2.1 had the ability to
> chop a touch up into pieces during replay, the problem could be solved.
> 
> This thread, as well as talking some more with Peter, makes me think we are
> pretty close. The things I see we need to do is:
> 
> 1. Add the ability to limit finger grouping to areas in grail.
> 
> The separation into finger groups is already there, but it is only based on the
> incoming data. Adding the ability to split the fingers into groups based on
> domains could be achieved by an api callback.
> 
> The visible simplifications during natty would be more expressive power to the
> application. In particular, the ability to have a dedicated scroll area. It
> would also enable the possibility to perform all event propagation from within
> the same process, although we would not push this scenario at all.
> 
> 2. Add the ability to split touches in XI2.1
> 
> The passive grab mechanism needs to be able to gracefully handle consumption of
> touches, such that touch-up and touch-down events are inserted appropriately if
> the consumer of a touch changes.

urgh. wouldn't you then get random touch down/up events on the screen,
depending on which application is currently listened to, and completely
disconnected from the user's interaction?

Cheers,
  Peter
 
> 3. Skip the tentative events in XI2.1
> 
> From Peter's explanation, it seems the original reason for this was to not run
> out of buffer space. Since the semantics is still unclear (just follow this
> thread for an example), it is probably best pushed to future expansions.
> 
> So, Gentlemen, what do you think of this?
> 
> Cheers,
> Henrik
>