Tangible interfaces: Collecting gestural and touch patterns
This transcript of an interview of Dan Saffer about his manifesto for gestural patterns for touch interaction is very pertinent. It’s mostly about this wiki resource which aims at collecting and disseminating gestural interface information and patterns, such as found on such devices as the iPhone and Wii (following a discussion Adaptive Path’s blog).
Some excerpts of this interview:
“How do you document this gesture where I’m sweeping my hand across the screen?” (…) This is our generation’s drag and drop.”
(…)
I felt it was a really important thing for interaction designers to be doing because, otherwise, we’re going to start to end up with a thousand different ways of turning on my TV where it’s like, “Is this the Microsoft TV where I have to snap my fingers three times or is it the Apple one where I twirl around in a circle?”
(…)
one of the nice things about having it be in a completely digital medium is that one of the problems with gestures is certainly documenting them. How do you describe something that’s not very ambiguous? It’s awfully difficult with words to describe gestures or even in diagrams to describe gestures.So having the ability to eventually put up movie clips showing this as a pattern with people moving their forefinger and thumb apart, for instance, having that kind of rich experience would be really nice on the website.“
(Picture taken from a wii game usability test I ran few months ago)
The examples he gives revolves around the Wii or the iPhone:
“The Wii certainly is very much about sort of movement in space. You’re not really touching anything except the controller. You’re kind of indirectly using a gesture. With the touch screen on the iPhone and other things, your fingertip is actually touching the device that you’re manipulating. So there is this gradation there.“
Why do I blog this? this is indeed an interesting issue, how you describe these movements? can we have a grammar (i.e. a set of patterns). This has some tight connections with a project I am involved in that tries to map the wiimote and nunchuk movements of existing games in a database, this will then allow to analyze them and document their relative importance.
November 4th, 2007 at 4:34 pm
I am not quite convinced with the Wii and iPhone example.
1) In both examples you *are* touching somenthing; you are touching the
input interface (the controller). A better example for not touching anything would have been, for instance, Kick Ass Kung-Fu, or any other application of computer vision.
2) In the iPhone case, input and output interfaces are overlaid (the multi-touch surface is overlaid to the screen), giving you the feeling you’re actually touching the virtual representations of data. As for the wiimote, you’re most of the time (but not all the time) using the a proxy for a real object (sword, gun, tennis racquet, kitchen knife) and gesturing with it.
So I really don’t get the sense of “not really touching anything except the controller” and “indirectly using a gesture”. To me it just illustrates the lack of a solid taxonomy for interaction.
And yes, let’s document gestures! And let’s document interface technologies and their application! We need taxonomies and design patterns for further use - and also to help poor students of Interface Design like me to be coherent when developing their thesis
November 6th, 2007 at 5:17 pm
I’d go along with Tiago questioning the future of gestural interface (where you don’t touch anything). I would see basicaly two “rules” of interaction design that would advocate against it :
- Gestural interfaces are not self-revealing : you don’t get to know what are your options and how to get to them. It’s like the difference between some command line and a visual interface.
- Gestural interfaces don’t provide feedback : with the large number degree of freedom you have, not having feedback would probably make it difficult to replicate precise gestures. To overcome this problem you have to define gestures that are “far away” (according to some gesture space metric) so that they cannot be misinterpreted, and thus reducing dramatically the expressiveness of these many degrees of freedom.
So it is probably more efficient to harness properly some space of limited expressiveness using a device that can provide both self-revealing and feedback capability.
November 6th, 2007 at 8:42 pm
Agreed! I fully agree with these statements, they are obvious limits. But this does not mean that they’re not relevant in some context (to be defined by user research)
November 7th, 2007 at 2:36 pm
Interfaces tangibles : à quoi nos gestes vont-ils servir ?…
Pour Dan Saffer, designer chez Adaptive Path, les designers d’interactions sont en passe de définir les principes fondateurs de nouveaux outils qui risquent d’être aussi importants que ceux définis dans les années 60 et 70 - et que nous…
November 19th, 2007 at 5:42 am
[…] Via Nicolas Nova. […]