multi-touch-dev team mailing list archive

[Rafi Rubin <rafi@xxxxxxxxxxxxxx>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/08/10 12:49, Stephen M. Webb wrote:
> On Fri, 2010-10-08 at 09:40 -0600, Duncan McGreggor wrote:
>>
>> I believe Stephen already has some changes he wants to make to this
>> proposed spec, so expect updates :-)
> 
> I'm thinking of radical changes, in fact.  This came out of the XDS
> decision to move gesture recognition into the client side.
> 
> Before I go to far, I want to give a brief description of what I think
> the 2.0 API should be like to see if this jives with everyone.
> 
> Geis 2.0 will have two ends:  an input end and a recognition end.
> 
> The application or toolkit will receive input events from, well,
> wherever.  XI2.1 in the case of X11, or native inputs in the case of a
> framebuffer-based installation (geis should not depend in any way on X).
> To that end, the application or toolkit will need to get an input cooker
> from geis, one per input device I imagine, and feed the raw input events
> into the cooker in its event loop.  The cooker transforms the raw input
> events into internal geis input events, and may require further inputs
> such as coordinate mappings depending on the requirements.

Just to be clear, are you talking about 1 central geis process per input device,
or one per device for each client?

> The application or toolkit will then have to poll the geis event loop.
> This drives outstanding cooked input events through the recognizer and
> results in zero or more events being output.  The possible events
> include (1) preview events -- the so-called tentative events, (2)
> gesture events, just like in geis 1.0, (3) touch events, for touches
> that are not recognized as part of a gesture, and (4) user-defined
> events, discussed below.  The application or toolkit can do what it
> likes with these events.

So if my understanding is correct, you prepare "proto-gestures" in the core geis
space, and then process that into the more concrete events in the client.

I like the idea of doing more computation in one space to aid in optimization.
Of course that gets messy if you want clients to be able to use arbitrary code
to interpret gestures.  This seems like a good balance.

What's the failure mode when a process stalls?  Will geis 1 and 2 react differently?

> I would create a default recognizer built on utouch-grail, but the next
> step (geis 2.1?) would be to provide a programmable recognizer, somewhat
> akin to the shaders available in 3D graphics rendering pipelines.
> Applications can load programmable gesture recognizers dynamically, and
> can have different recognizers for different screen regions or input
> devices.  The programmable recognizers can emit the user-defined events
> mentioned above.

A good comparison, I wouldn't be too surprised to hear that someone actually
does play with a gpu to accelerate gesture recognition :)

Intel recently announced Atom's with an in package Altera FPGA, that would be an
obvious target for low power, low latency, hardware accelerated gesture
processing.  Of course that only makes sense if people have relatively expensive
gestures they want to use.

> There are still some details I need to work out before I can formalize
> this more, but I would value some feedback on whether this is sane or
> not.

As long as you don't need to drastically increase the effort for a client trying
to get the functionality it had with 1.0 in the new api, and you see a clear
path to implementation then I think this is quite sane.

I'm happy to see you moving in this direction.

Rafi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJMstPLAAoJEPILXytRLnK2qvYP/RBN2SGGaOEeYKNarFf/ehES
R6S2Gg0jvDLlcjX4iHsazVGV0a8UtY9+x+GmJAUF4ej4u5N5R940O9GNvR1vBHG+
0nPxpR9rMYKxiqSZz5NDNi+Se9PmmYIZD7TnCn0xT4K8lIZXjiCj8kQt+b6p3pYn
1bEQV+PHRG5kb69wqS/L1LnKElTFo47EjrXyBuC8LkdyxTr+8XZwzv2SqssaELDB
7Y1ghr01HKSlFaD2heb/wpgyqMgIniWNhEQTiD5Y45CkZ0anCGBJhQTgcdSUhi1s
4rYnnkjRCGOzVpdqBXtGFUg/siwf7RL8UUmlRMutRRpKloFskpAyaeq/d4KGA1EF
R8f/DyJ0pQ8liSdzUD1Bk8j7YDbaHLDOPdBo+o2mmhf+4kxdbigvCXByeUcrtAa/
BsUuVo5LxQfQ/kkdJjezuDO4ZsxrKEKDDRFO/79ifn667f/8HpAGa6myKbNwvQ9P
0oWvlIB93RcK3lBfxatUlxlu9GuUGTOl0unmvNTRBUsL7vSRytU3FRNOWd6B7Byx
D52Lyd+T/z7vNVEkOGW7uZ57SI7F+YSY24JY+IePz4UMGbGNN6+Z+rq0I7G9deW1
hzWoLDy5vv3JsUfJ80LV380KPgUJmOoVEyc8ExQZ26dSyRv8YPD6rb5YsnGIaWzA
/HQaEmHj0KfhhXCxkWU+
=37Yj
-----END PGP SIGNATURE-----