multi-touch-dev team mailing list archive

[Chase Douglas <chase.douglas@xxxxxxxxxxxxx>]

On Tue, 2010-09-07 at 15:50 +0100, John Lea wrote:
> On 07/09/10 14:41, Chase Douglas wrote:
> > On Tue, 2010-09-07 at 10:26 +0100, John Lea wrote:
> >    
> >> On 02/09/10 21:32, Chase Douglas wrote:
> >>      
> >>> 1. The spec says, "Performing the 'greedy' trigger gesture on top of an
> >>> application that does not support a greedy state will produce no
> >>> effect." How will this be determined?
> >>>        
> >> If the application does not have a greedy state nothing will happen in
> >> response to a user performing the greedy gesture over the applicaiton.
> >> This is simply the null case; the greedy gesture has no effect on
> >> applications that do not specifically support greedy interaction.
> >>
> >>      
> >>> What if inkscape understands
> >>> multitouch but not gestures? Will Unity not unlock it because inkscape
> >>> hasn't subscribed to three or four finger gestures?
> >>>
> >>>        
> >> If inkscape has a greedy state, the greedy gesture will toggle the
> >> greedy state on or off.  If inkscape does not have a greedy state it
> >> will be limited to only receiving one and two finger multi-touch input
> >> and one and two finger gestures.
> >>
> >> I'm unsure about your comment regarding inkscape supporting only
> >> multi-touch but not gestures - I think we will want inkscape to support
> >> gestures for moving the view port, resizing and rotating objects,
> >> etc...?  For an application to officially work and comply with the Unity
> >> Touch platform it has to support gestures.  I can see that some
> >> applications may make use of gestures without using multi-touch but I
> >> can't see it happening the other way around.   However I may be wrong,
> >> are there any specific use cases you have in mind?
> >>
> >> Also see section 3.2.4 on protected gestures for some more info.
> >>      
> > I think there is a very good possibility that some applications will
> > support multitouch without gestures. Multitouch input is the bare-metal
> > approach. In the long term, we want people to be using our gesture
> > engine when it is useful, but what happens if an application wants to
> > handle gestures in its own way? I'm really thinking of the 1% of
> > applications no one thinks about when they make their toolkits and
> > libraries. I don't want to harm their usability, nor users' patience,
> > when certain applications are broken when run on Ubuntu.
> >    
> 
> Apps can receive one and two finger multi-touch input, but to go beyond 
> that they need become greedy.  This septation is unavoidable because of 
> our decision to reserve multi-touch events using more than two fingers 
> for system functions.  Allowing apps to grab and interrupt Screen and 
> Unity level gestures without following our greedy conventions would 
> break the overall Unity user experience. So if the app authors want 
> their apps to work under Unity they will at a minimum have to support 
> the greedy state.

To be fair, unity could relinquish multitouch control to all windows
that are unlocked. This does not need to be limited only to those
clients that register for three or more touches. Is it a big deal if a
non multitouch aware window is unlocked? The user will be given an
indication that it's been unlocked, so what's the harm? I think there's
a pragmatism vs. idealism argument here, and I'm probably falling on the
pragmatic side so we can ensure apps work well everywhere without
requiring a lot of effort on the part of individual app developers.

> >>> 4. I worry about the undo gesture and it's implementation/usage. Why not
> >>> just queue up multitouch events while a three-finger hold is being
> >>> performed, and only sending them to the application when the touches are
> >>> determined not to be a greedy trigger? This would remove the requirement
> >>> for a drawing application to be gesture aware.
> >>>
> >>>        
> >> This would add user visible latency which we are trying to avoid at all
> >> costs.  Low latency and perceived responsiveness are really important to
> >> the user experience.
> >>      
> > Latency is an issue, but the semantics of this approach puts a lot on
> > the plate of developers of applications. They now have to add a buffer
> > to maintain undo state for multitouch. It shouldn't be integrated with
> > the normal undo state because then a user might accidentally redo the
> > three-finger hold.
> 
> Humm, I might be wrong but I don't think that the user pressing redo 
> would redo the three finger hold any more that it would redo the mouse 
> pointer moving to 324x790px on the screen and repeat the right mouse 
> button press.  I think the undo-redo history should store the actions 
> performed, not the exact events that triggered them??

In some scenarios you are attempting to undo state that is in the
application. Take Inkscape as an example again. Stuff has been painted
to the canvas, so the application needs to undo that painting.

> > Or worse, if a user has undone a few steps, then does
> > a three-finger hold, the undone steps will be overwritten by the hold
> > gesture events.
> >    
> 
> Again the three finger hold itself should not impact the undo-redo 
> history, only the actions performed as a result of the interaction 
> should be stored?  As this is the current behavior of undo/redo I don't 
> think it will cause any problems.

Here's an example of what I'm thinking of in a drawing application:

1. User creates a stroke
2. User decides stroke is wrong
3. User hits ctrl-Z to undo
4. User holds three fingers down to enter greedy state
  a. Inkscape starts painting on the canvas, which overwrites the redo
buffer created by the stroke in step 1.
  b. Greedy state is entered, undo gesture initiated
  c. Inkscape undoes three finger painting, three finger painting events
end up in redo buffer

At the end of all this, the user expects that the stroke from step 1 is
in the redo buffer and greedy state has been entered. Instead, the user
finds that greedy state has been entered, but the three finger touches
that were painted are in the redo buffer instead of the stroke from step
1.

To properly handle undo events for reserved gestures, the application
would need to create a new buffer just for gesture undo events.

> > We have managed to be very responsive to multitouch input when
> > recognizing gestures. I think we can be just as responsive when
> > recognizing a hold without needing to use undo events. Would you be
> > amenable to trying to support reserved gestures without any undo events,
> > and then adding undo events back in if it doesn't work?
> >    
> Think about the case of a finger painting program.  The user starts 
> painting a three finger anti-clockwise circle (one of the ideas for the 
> undo gesture).  It would be really strange if the finger painting 
> program didn't leave paint trails immediately behind the three fingers.  
> So when the user finishes performing the anti-clockwise circle and the 
> touch events are recognized as the undo gesture, and a undo command is 
> sent to the finger painting program.  This undoes the circle that has 
> just been painted (confusingly, as this is the undo gesture, another 
> undo command is also sent to undo the action performed before the user 
> started performing the gesture).  ;-)

I'm not sure I agree that it's less confusing to draw the arcs as an
undo is being recognized. What I have been thinking is the following:

1. You start the three finger rotation for undo
2. As you continue the motion, an indicator on screen says (in diagram
not words of course): "It looks like you're doing an undo. If you
continue up to this point, an undo command will be sent. If you stop, or
continue past this point etc, your events will not be interpreted as
gestures and will be sent to the application."

Otherwise, you'll end up seeing stuff get drawn, and without any
indication the stuff disappears. And more: my last stroke disappeared
too!

iOS gets away with this by using the accelerometers to register a shake
as an undo event :). 

-- Chase