multi-touch-dev team mailing list archive

[Chase Douglas <chase.douglas@xxxxxxxxxxxxx>]

On Fri, 2010-10-15 at 17:23 +0200, Henrik Rydberg wrote:
> On 10/14/2010 09:39 PM, Chase Douglas wrote:
> > On Wed, 2010-10-13 at 20:43 +0200, Henrik Rydberg wrote:
> >> On 10/13/2010 06:03 PM, Chase Douglas wrote:
> >>>> Good point. Would you mind going through, step-by-step, the case where we start
> >>>> with two fingers inside a "window", scroll a bit, then add another finger in a
> >>>> different window, and perform a three-finger pinch?
> >>>
> >>> There's a few context questions we need to ask first (I'm trying to get
> >>> at what your asking, maybe I'm going off in the wrong direction though):
> >>>
> >>> 1. Is a three finger pinch a system-wide gesture? I'll assume yes. Are
> >>> two finger gestures system-wide gestures? I'll assume no.
> >>>
> >>> 2. Should a two-to-three finger movement that is not immediate, by which
> >>> I mean this isn't a three finger movement from the start where the first
> >>> couple of device touch frames caught only two fingers, trigger a three
> >>> finger gesture? I don't believe we've explicitly answered this anywhere
> >>> yet, but I think not. A three finger gesture must be initiated with
> >>> three fingers within a short period of time on the order of 10s of
> >>> milliseconds (I'm leaving aside the case of gesture continuation).
> >>
> >>
> >> I tend to agree, although that is all about policy. I was intending this example
> >> to trigger a global gesture, since it makes the example more difficult.
> >>
> >>> Given these assumptions, I think the expected result is that the
> >>> scrollable region continues to scroll according to the two fingers over
> >>> it, and the single finger in the other window does whatever it should in
> >>> that window.
> >>
> >>
> >> Right, that is the easy path. So what about the one which triggers a global gesture?
> > 
> > I'm not sure we've defined a mechanism that would work in that case.
> > What you would need is the ability to peek at all touch events
> > individually and pick and choose which ones you want to keep and which
> > you want to replay. The reason this gets dicey is what does the client
> > do? Imagine the following:
> > 
> > 1. Touch begins
> > 2. WM replays touch events one at a time
> > 3. Application gets touch begin, touch motion events
> > 4. WM consumes some events
> > 5. WM replays more events
> > 6. WM consumes events
> > 7. Touch ends
> > 
> > Issues:
> > 
> > * Does application get touch begin events when touch physically begins
> > if WM starts consuming all events? Or does it only receive a begin event
> > once the first replayed event is sent?
> > * Is application notified when WM starts consuming events again?
> > * What happens when events are replayed in step 5? It may seem that the
> > touch moved from one are of the screen to another instantaneously when
> > in reality the touches in between were consumed by the WM.
> > * If the WM consumes the touch end, does the application then receive
> > the touch end?
> > 
> > I think this use case is opposite to good UI design, and I think the
> > plumbing to make it possible opens up a big can of worms. X doesn't
> > support any replay mechanism like this for even legacy pointer events.
> > We seem to all agree that the original example bore the correct outcome,
> > so I'm hesitant to continue down this path until we find a scenario that
> > would require it.
> > 
> > There's also nothing that prevents us from adding support for this in
> > the future if we find we need it.
> 
> 
> Thanks for showing the implications, Chase. I think it illustrates well that MT
> and Gestures cannot be separated.
> 
> If we think of MT events as separate single-finger drags, MT becomes a subset of
> gestures. In grail, a N-finger gesture overrides all other gestures that can be
> performed with M < N fingers. This includes one-finger drags. Going back to your
> example, the natural interpretation is that when the three-finger gesture
> starts, the two single-finger drags end. Perhaps if XI2.1 had the ability to
> chop a touch up into pieces during replay, the problem could be solved.

I'm not sure I follow. Above, I noted all the issues with chopping up
touches during replay. That doesn't seem to be a reasonable solution to
me, at least not without more details if you envision some solution to
the issues.

> This thread, as well as talking some more with Peter, makes me think we are
> pretty close. The things I see we need to do is:

Hrm, based on my understanding of things, I think you are venturing
further apart from me, and what I thought Peter was thinking too :).

> 1. Add the ability to limit finger grouping to areas in grail.
> 
> The separation into finger groups is already there, but it is only based on the
> incoming data. Adding the ability to split the fingers into groups based on
> domains could be achieved by an api callback.

Why not just have the toolkit/application pass touches for areas into
grail separately? I'm not seeing why we need an api for regions in
grail/geis?

My thought was to have grail heuristically determine touch groupings
within a set passed to it by the application. If grail sees a large area
and touches are far apart, it can try to group them appropriately.
There's no region boundaries though.

> The visible simplifications during natty would be more expressive power to the
> application. In particular, the ability to have a dedicated scroll area.

This is more easily done through having the toolkit define a widget for
the scroll area and passing MT events in that area to grail. I'm not
sure what providing a region to grail would accomplish beyond this.

> It would also enable the possibility to perform all event propagation from within
> the same process, although we would not push this scenario at all.

Are you talking about propagation between X "windows"? If so, I thought
that was deemed ill advised by myself and Peter.

> 2. Add the ability to split touches in XI2.1
> 
> The passive grab mechanism needs to be able to gracefully handle consumption of
> touches, such that touch-up and touch-down events are inserted appropriately if
> the consumer of a touch changes.

Doesn't this just open up the can of worms I described above?

> 3. Skip the tentative events in XI2.1
> 
> From Peter's explanation, it seems the original reason for this was to not run
> out of buffer space. Since the semantics is still unclear (just follow this
> thread for an example), it is probably best pushed to future expansions.

Without tentative events, how will you provide low latency UI feedback
to the user?

The semantics of tentative events seems clear to me, what issues do you
see? In fact, we haven't really discussed the alternative of queuing up
grabbed events for serial replaying, but I think that has even more
issues because of event ordering. If you go with tentative events, you
can send all the events in the correct order the first time. If you
queue up events, you have to worry about how device change events (known
elsewhere as DCEs) in the XI 2 spec are handled.

> So, Gentlemen, what do you think of this?

I feel like I'm really missing something, because most of the thoughts
above are opposite to what I've been thinking all along. Maybe I'm just
not reading them right?

-- Chase