multi-touch-dev team mailing list archive

[John Lea <john.lea@xxxxxxxxxxxxx>]

On 07/09/10 14:41, Chase Douglas wrote:
> On Tue, 2010-09-07 at 10:26 +0100, John Lea wrote:
>    
> > On 02/09/10 21:32, Chase Douglas wrote:
> >      
> > > 1. The spec says, "Performing the 'greedy' trigger gesture on top of an
> > > application that does not support a greedy state will produce no
> > > effect." How will this be determined?
> > >        
> > If the application does not have a greedy state nothing will happen in
> > response to a user performing the greedy gesture over the applicaiton.
> > This is simply the null case; the greedy gesture has no effect on
> > applications that do not specifically support greedy interaction.
> >
> >      
> > > What if inkscape understands
> > > multitouch but not gestures? Will Unity not unlock it because inkscape
> > > hasn't subscribed to three or four finger gestures?
> > >
> > >        
> > If inkscape has a greedy state, the greedy gesture will toggle the
> > greedy state on or off.  If inkscape does not have a greedy state it
> > will be limited to only receiving one and two finger multi-touch input
> > and one and two finger gestures.
> >
> > I'm unsure about your comment regarding inkscape supporting only
> > multi-touch but not gestures - I think we will want inkscape to support
> > gestures for moving the view port, resizing and rotating objects,
> > etc...?  For an application to officially work and comply with the Unity
> > Touch platform it has to support gestures.  I can see that some
> > applications may make use of gestures without using multi-touch but I
> > can't see it happening the other way around.   However I may be wrong,
> > are there any specific use cases you have in mind?
> >
> > Also see section 3.2.4 on protected gestures for some more info.
> >      
> I think there is a very good possibility that some applications will
> support multitouch without gestures. Multitouch input is the bare-metal
> approach. In the long term, we want people to be using our gesture
> engine when it is useful, but what happens if an application wants to
> handle gestures in its own way? I'm really thinking of the 1% of
> applications no one thinks about when they make their toolkits and
> libraries. I don't want to harm their usability, nor users' patience,
> when certain applications are broken when run on Ubuntu.
>    

Apps can receive one and two finger multi-touch input, but to go beyond 
that they need become greedy.  This septation is unavoidable because of 
our decision to reserve multi-touch events using more than two fingers 
for system functions.  Allowing apps to grab and interrupt Screen and 
Unity level gestures without following our greedy conventions would 
break the overall Unity user experience. So if the app authors want 
their apps to work under Unity they will at a minimum have to support 
the greedy state.

However we want to very strongly encourage apps to use our gesture 
library.  Without all apps using the same gesture library the gesture 
interpretation behavior will end up varying widely leading to a bad user 
experience.  UX consistency is especially critical when working with 
gestures because of the innate challenge of discoverability.

I understand your concerns, but hopefully by being first to market we 
can set the standards that others will follow.  Also our more or less 
explicit design target is to deliver a touch interface that leapfrogs 
Apple and Android.  To do this we have to stakeout a strong leadership 
position in terms of the touch UX, and we won't be able to deliver a 
competitive platform if a number of the apps we ship behave 
inconsistently with each other.

> This is something we can leave for later though. No one really supports
> multitouch at an application level yet, so we shouldn't be breaking
> anything.
>    

My hope exactly ;-)  This is also why I am so pleased that we are doing 
everything in public.

>    
> > > 4. I worry about the undo gesture and it's implementation/usage. Why not
> > > just queue up multitouch events while a three-finger hold is being
> > > performed, and only sending them to the application when the touches are
> > > determined not to be a greedy trigger? This would remove the requirement
> > > for a drawing application to be gesture aware.
> > >
> > >        
> > This would add user visible latency which we are trying to avoid at all
> > costs.  Low latency and perceived responsiveness are really important to
> > the user experience.
> >      
> Latency is an issue, but the semantics of this approach puts a lot on
> the plate of developers of applications. They now have to add a buffer
> to maintain undo state for multitouch. It shouldn't be integrated with
> the normal undo state because then a user might accidentally redo the
> three-finger hold.

Humm, I might be wrong but I don't think that the user pressing redo 
would redo the three finger hold any more that it would redo the mouse 
pointer moving to 324x790px on the screen and repeat the right mouse 
button press.  I think the undo-redo history should store the actions 
performed, not the exact events that triggered them??

> Or worse, if a user has undone a few steps, then does
> a three-finger hold, the undone steps will be overwritten by the hold
> gesture events.
>    

Again the three finger hold itself should not impact the undo-redo 
history, only the actions performed as a result of the interaction 
should be stored?  As this is the current behavior of undo/redo I don't 
think it will cause any problems.

> We have managed to be very responsive to multitouch input when
> recognizing gestures. I think we can be just as responsive when
> recognizing a hold without needing to use undo events. Would you be
> amenable to trying to support reserved gestures without any undo events,
> and then adding undo events back in if it doesn't work?
>    
Think about the case of a finger painting program.  The user starts 
painting a three finger anti-clockwise circle (one of the ideas for the 
undo gesture).  It would be really strange if the finger painting 
program didn't leave paint trails immediately behind the three fingers.  
So when the user finishes performing the anti-clockwise circle and the 
touch events are recognized as the undo gesture, and a undo command is 
sent to the finger painting program.  This undoes the circle that has 
just been painted (confusingly, as this is the undo gesture, another 
undo command is also sent to undo the action performed before the user 
started performing the gesture).  ;-)

Can you think of another way to handle this use case?

> What I'm afraid of with undo and the greedy state enforcement is that
> application developers will see things work fine on other distros and
> either not care about Ubuntu or, even worse, paint ubuntu in a bad light
> because we are breaking their applications. Right or wrong, there are
> many people out there who do not believe we have any right to set
> standards because we don't post them to freedesktop.org. We want to show
> them that we have great ideas and make them salivate at the opportunity
> to support gestures in their applications. Breaking their current
> implementations and forcing them to write a lot of new code just so it
> works right in Ubuntu may push them away instead.
>    

There is a balancing act to perform here, we obviously want to comply 
with as many of the existing open standards as possible.  But part of 
Unity is about creating a platform, and to take full advantage of the 
Unity platform apps will have to be either specifically written or 
customized.  We are attempting to overtake Android and iOS, and we won't 
be able to do that while staying fully within the bounds of existing 
conventions, especially in an emerging area like touch.

cheers,
John

--
John Lea | User Experience Architect
Canonical  www.canonical.com | Ubuntu  www.ubuntu.com
27th Floor, 21-24 Millbank Tower, London, SW1P 4QP
Tel: +44 (0) 20 7630 2415 | Email: john.lea@xxxxxxxxxxxxx