multi-touch-dev team mailing list archive

[Chase Douglas <chase.douglas@xxxxxxxxxxxxx>]

On Wed, 2010-10-13 at 11:41 +0200, Henrik Rydberg wrote:
> On 10/12/2010 10:29 PM, Chase Douglas wrote:
> > On Tue, 2010-10-12 at 21:27 +0200, Henrik Rydberg wrote:
> >> There is no argument that client-side grail solves client-side event
> >> propagation. The argument is whether supporting the linkage between the two
> >> propagation models is a good idea.
> >>
> >> I appreciate that the solution Peter and yourself have arrived at is navigating
> >> delicately in a somewhat jungleish environment. However, duplicating the window
> >> event propagation on the client side may very well be a simpler solution.
> >> Implementing get_clients() would not be much more complicated, would it?
> >> Stephen's ideas are starting to make sense to me. :-)
> > 
> > If you went this route, where toolkits and apps listen directly to the
> > kernel and are assumed to propagate events between them appropriately,
> > you'll have issues once you start mixing in legacy apps and toolkits
> > that listen for this data from X.
> 
> We would of course have a single point of event propagation, so "between them"
> would not enter the equation. Since there is no gesture data from X, there is no
> mixing. And there are no applications listening to MT events from X either,
> since it does not exist yet. Thus, there are no real issues here, except in what
> direction we want to go.

Ok, in this case I'm having a hard time seeing how this is different
than putting the cross-"windows" MT propagation logic in the X server
and having gestures and widget MT propagation client side. We can't put
gesture recognition anywhere other than in each individual applications
because each application may be written with a different toolkit. The
recognition regions are only known within the process space of
individual clients.

> > Also, in order to perform proper propagation between "windows", each
> > application will need to query the window stack on every new input event
> > to be sure it is allowed to handle the input.
> 
> 
> Definitely not. The point is that the process doing propagation can have the
> full picture because it is placed where all information is available.

The X process space is the only place where window position information
is available internally.

> Here is my current list of problems:
> 
> 1. Mechanism for multi-user input.
> 
> Which fingers are supposed to act together can be seen as a mix of physical
> constraints (no way these two fingers are from the same person; nah, that's an
> elbow, never mind that; i only consider moving fingers, resting ones mean
> something else) and user interface (put fingers here and they are considered
> together; put them here and they are not; oh, but that's a global gesture, I
> want that after all). For this problem, at the very least, event construction
> requires the knowledge of the various areas constraining gestures, in addition
> to logic for decomposing the physical input. It is not even clear whether this
> is a solvable problem.

I see two mechanisms:

1. Gesture regions (individual widgets/windows) subdividing an MT
touchscreen surface. If there are two users with two separate
applications open, then gestures in each application window are already
confined.

2. Heuristics for grouping touches within a single region. If the region
is large and multiple touches are far apart, then we can infer they are
from different hands or people. These heuristics can be part of the
gesture recognizer. If XI 2.1 passive grabbing is used for system-wide
gestures, one group of touches may be recognized as a gesture and
consumed while other touches are "replayed" to other clients.

Of course the heuristics are to be determined, but I don't think we'll
be able to work on them before we need to have single-user gestures
working properly.

> 2. Mechanism for single-user gestures
> 
> This one is pretty much solved. Combine the given set of fingers into gestures.
> The problem of whether MT data should be passed through does not really enter
> here, in the sense that by stating that fingers should be considered jointly,
> the individual fingers loose meaning.
> 
> 3. Mechanism for MT data
> 
> This one is solved several times over by kernel, raw touch data, and XI2.1.
> Taking gesture out of the picture makes this problem easy.
> 
> It is clear that the part still unsolved has to do with multi-user input. I
> include the question whether a gesture should in fact be seen as a set of
> individual fingers. Combinatorics, projections, greedy versus global... I think
> we need a handle on this problem before deciding what direction to take.

Certainly skepticism is necessary as we formulate a solution, so thanks
for trying to highlight issues. In my mind I don't see any real issues
though. Do you have any scenarios where the proposed mechanisms break
down?

-- Chase