multi-touch-dev team mailing list archive

[Henrik Rydberg <rydberg@xxxxxxxxxxx>]

On 10/12/2010 10:29 PM, Chase Douglas wrote:

> On Tue, 2010-10-12 at 21:27 +0200, Henrik Rydberg wrote:
>> On 10/12/2010 06:53 PM, Chase Douglas wrote:
>>> On Tue, 2010-10-12 at 16:31 +0200, Henrik Rydberg wrote:
>>>> On 10/12/2010 03:49 PM, Chase Douglas wrote:
>>>>> Going along with this, event propagation for widgets in a "window" is
>>>>> performed in the toolkit. When using modern toolkits, X event
>>>>> propagation is only used to propagate events between application
>>>>> "windows" on screen.
>>>>>
>>>>> (Here I use quotes around "window" to refer to what a normal user would
>>>>> call an application window, in contrast to the X definition of a
>>>>> window.)
>>>>>
>>>>> This leads us to the gesture recognition mechanism Peter and I have been
>>>>> discussing. Since we can't perform gesture recognition in the server, we
>>>>> have to potentially perform recognition twice: first at the system-wide
>>>>> level, then potentially at the application level if no system-wide
>>>>> gestures were recognized. One method for doing this is XI 2.1 passive
>>>>> grabbing.
>>>>
>>>>
>>>> It seems to me another solution would be to route the events directly to the
>>>> client side and use a more modern event propagation model.
>>>
>>> This sounds like another client-server model, but with a different
>>> server now. I'm not sure how this solves the event propagation issues
>>> with client side toolkit rendering, and I only see it duplicating what
>>> could be done more naturally in the window server input layer.
>>
>>
>> There is no argument that client-side grail solves client-side event
>> propagation. The argument is whether supporting the linkage between the two
>> propagation models is a good idea.
>>
>> I appreciate that the solution Peter and yourself have arrived at is navigating
>> delicately in a somewhat jungleish environment. However, duplicating the window
>> event propagation on the client side may very well be a simpler solution.
>> Implementing get_clients() would not be much more complicated, would it?
>> Stephen's ideas are starting to make sense to me. :-)
> 
> If you went this route, where toolkits and apps listen directly to the
> kernel and are assumed to propagate events between them appropriately,
> you'll have issues once you start mixing in legacy apps and toolkits
> that listen for this data from X.


We would of course have a single point of event propagation, so "between them"
would not enter the equation. Since there is no gesture data from X, there is no
mixing. And there are no applications listening to MT events from X either,
since it does not exist yet. Thus, there are no real issues here, except in what
direction we want to go.


> Also, in order to perform proper propagation between "windows", each
> application will need to query the window stack on every new input event
> to be sure it is allowed to handle the input.


Definitely not. The point is that the process doing propagation can have the
full picture because it is placed where all information is available.


>> The problem I have with passive grabs is that it uses tentative events as a
>> solution to the round-trip problem involved in accepting MT events. In my mind,
>> tentative events should be used to show something different from normal events,
>> something that makes the user understand their volatile nature. In this respect,
>> inhibiting pointer motion during two-finger scroll is quite different from user
>> feedback during a rotation gesture.
> 
> I think we are thinking of two different types of events. The XI 2.1
> passive grabbing provides for tentative device events. You are talking
> about tentative event actions? Tentative actions are built upon either
> tentative or active device events, but we need tentative device events
> to be able to potentially handle the same events either system-wide or
> at the application level.


This is not true. We need to gather enough information to be able to make a
decision on what should happen. At best, tentative events is a way of sorting
this out using a distributed decision mechanism. There are a number of ways we
could formulate this.


>> I hope this does not sound too discouraging - I am simply trying to turn some
>> stones here. If you already turned them, I am sure you will tell me. :-)
> 
> Heh, that's exactly what I'm hoping for, for you to turn more stones. I
> don't think any of us would agree that XI 2.1 passive grabs are elegant
> or clean. I certainly don't want to help implement them, but them's the
> breaks :). Please keep digging if you think you can find a better
> solution!


Here is my current list of problems:

1. Mechanism for multi-user input.

Which fingers are supposed to act together can be seen as a mix of physical
constraints (no way these two fingers are from the same person; nah, that's an
elbow, never mind that; i only consider moving fingers, resting ones mean
something else) and user interface (put fingers here and they are considered
together; put them here and they are not; oh, but that's a global gesture, I
want that after all). For this problem, at the very least, event construction
requires the knowledge of the various areas constraining gestures, in addition
to logic for decomposing the physical input. It is not even clear whether this
is a solvable problem.

2. Mechanism for single-user gestures

This one is pretty much solved. Combine the given set of fingers into gestures.
The problem of whether MT data should be passed through does not really enter
here, in the sense that by stating that fingers should be considered jointly,
the individual fingers loose meaning.

3. Mechanism for MT data

This one is solved several times over by kernel, raw touch data, and XI2.1.
Taking gesture out of the picture makes this problem easy.

It is clear that the part still unsolved has to do with multi-user input. I
include the question whether a gesture should in fact be seen as a set of
individual fingers. Combinatorics, projections, greedy versus global... I think
we need a handle on this problem before deciding what direction to take.

Cheers,
Henrik