multi-touch-dev team mailing list archive

[Henrik Rydberg <rydberg@xxxxxxxxxxx>]

On 10/13/2010 04:14 PM, Chase Douglas wrote:
[...]
>>> If you went this route, where toolkits and apps listen directly to the

>>> kernel and are assumed to propagate events between them appropriately,
>>> you'll have issues once you start mixing in legacy apps and toolkits
>>> that listen for this data from X.
>>
>> We would of course have a single point of event propagation, so "between them"
>> would not enter the equation. Since there is no gesture data from X, there is no
>> mixing. And there are no applications listening to MT events from X either,
>> since it does not exist yet. Thus, there are no real issues here, except in what
>> direction we want to go.
> 
> Ok, in this case I'm having a hard time seeing how this is different
> than putting the cross-"windows" MT propagation logic in the X server
> and having gestures and widget MT propagation client side. We can't put
> gesture recognition anywhere other than in each individual applications
> because each application may be written with a different toolkit. The
> recognition regions are only known within the process space of
> individual clients.


This is under the assumption that we don't add regions to geis, or any other way
of obtaining windowing information from the toolkits.

>>> Also, in order to perform proper propagation between "windows", each
>>> application will need to query the window stack on every new input event
>>> to be sure it is allowed to handle the input.
>>
>>
>> Definitely not. The point is that the process doing propagation can have the
>> full picture because it is placed where all information is available.
> 
> The X process space is the only place where window position information
> is available internally.


We went half-way towards extracting that information once already, so we know it
is a bit cumbersome, but possible.

>> Here is my current list of problems:
>>
>> 1. Mechanism for multi-user input.
>>
>> Which fingers are supposed to act together can be seen as a mix of physical
>> constraints (no way these two fingers are from the same person; nah, that's an
>> elbow, never mind that; i only consider moving fingers, resting ones mean
>> something else) and user interface (put fingers here and they are considered
>> together; put them here and they are not; oh, but that's a global gesture, I
>> want that after all). For this problem, at the very least, event construction
>> requires the knowledge of the various areas constraining gestures, in addition
>> to logic for decomposing the physical input. It is not even clear whether this
>> is a solvable problem.
> 
> I see two mechanisms:
> 
> 1. Gesture regions (individual widgets/windows) subdividing an MT
> touchscreen surface. If there are two users with two separate
> applications open, then gestures in each application window are already
> confined.
> 
> 2. Heuristics for grouping touches within a single region. If the region
> is large and multiple touches are far apart, then we can infer they are
> from different hands or people. These heuristics can be part of the
> gesture recognizer. If XI 2.1 passive grabbing is used for system-wide
> gestures, one group of touches may be recognized as a gesture and
> consumed while other touches are "replayed" to other clients.
> 
> Of course the heuristics are to be determined, but I don't think we'll
> be able to work on them before we need to have single-user gestures
> working properly.
> 
>> 2. Mechanism for single-user gestures
>>
>> This one is pretty much solved. Combine the given set of fingers into gestures.
>> The problem of whether MT data should be passed through does not really enter
>> here, in the sense that by stating that fingers should be considered jointly,
>> the individual fingers loose meaning.
>>
>> 3. Mechanism for MT data
>>
>> This one is solved several times over by kernel, raw touch data, and XI2.1.
>> Taking gesture out of the picture makes this problem easy.
>>
>> It is clear that the part still unsolved has to do with multi-user input. I
>> include the question whether a gesture should in fact be seen as a set of
>> individual fingers. Combinatorics, projections, greedy versus global... I think
>> we need a handle on this problem before deciding what direction to take.
> 
> Certainly skepticism is necessary as we formulate a solution, so thanks
> for trying to highlight issues. In my mind I don't see any real issues
> though. Do you have any scenarios where the proposed mechanisms break
> down?


Good point. Would you mind going through, step-by-step, the case where we start
with two fingers inside a "window", scroll a bit, then add another finger in a
different window, and perform a three-finger pinch?

Cheers,
Henrik