The near future laboratory (Fabien Girardin and Julian Bleecker and myself) organize a workshop during LIFT08 about the failures of ubiquitous computing.

The workshop is called “Ubiquitous computing: visions, failures and new interaction rituals“:

“We propose to look backward and discusses why we have not reached what has been described in the last 5 years of ubiquitous computing. How might we criticize assumptions and build upon existing models and approaches to design in this context? Can we learn from the discrepancy between the utopia of ubicomp and its deployed reality?

The purpose is to generate debate about the design and integration of ubiquitous systems based on case studies proposed from workshop participants. Moreover, we want to open up a debate around the future of those systems as well as the adoption by a large user base.

The session will start by a short presentation by participants who will each have to describe in 2-3 minutes a ubiquitous computing system that failed and give reasons or causes for that.
…“

Why do I blog this? LIFT is always an enjoyable moments for workshops, let’s see what emerge out of that one.

Currently in Aix-en-Provence, at the School of Art where I’ve given a talk yesterday about tangible interfaces (a rerun from my GDC2007 presentation). The talk was part of a workshop called “Workshop Wiimote Hacking. The whole thing was about how to hack the Nintendo Wiimote and turn it into a tool that artists can employ. Students are involved in the process of adding new sensors as well as defining new sorts of usage. Thanks France Cadet for the invitation and Douglas for having taken time to discuss.

The discussion after the presentation revolved around:
- How the wiimote might have the potential to become a sort of standard in the living room as the controller. I personally don’t predict anything about this. However from current observations of practices, I do think that the Wii is much more than video game device and play an interesting role of multimedia platform in the living room (with photos, usage of personalized mii). In addition, the presence of very focused applications such as wii questions turns it into a platform where facebook-like small application can be played by the family.
- Why the game design ideas we have so far on the Wii are so conservative… which turned into why the video game industry is so conservative or how the whole economic system is so controlled by the marketing crowd that it’s difficult to go beyond usage of the wiimote as sword/steering-wheel/magic wand. This is a sort of over-statement but it’s actually close to the reality.
- How this work about tangible interfaces relate to my other work about mobile and pervasive gaming? we discussed the notion of granularity and how complex the game system is when the dimensions reach the city level. Much more than just gesturing in one’s own living room, using the city as a game field is complex for lots of reasons (technical, infrastructural, difficulty to have a continuous experience), etc.

Fortunately, there was also an exhibit about games as medium for artists to create and tweak digital worlds. Called gamerz02, there was a bunch of very curious projects.

The one that attracted my attention was Patch&KO (Antonin Fourneau and Manuel Braun): instead of using a joystick people can play Street Fighter 2 using a pachinko interface. In a sense, the player’s ability to control the character is disrupted by the semi-random movement of the metal balls. As the designers state, the player has to accept a loss of control. Slightly related was this Tictactoe played by robotic arms I blogged about the other day. In this case, what was explored was that the Tic Tac Toe is a curious game in which the only way to win is to rely on the opponent’s fatigue and loose of sight in the game. In the context of two mechanical arms played by a computer, the surest way to win is to avoid playing. Finally, I was also interested by Tchouri by Pascal Silondi, a sort of knife-based interface much more intriguing than always-seen magic wands. Some of the pieces there were maybe slightly easy-going and naive but the whole things made sense and I found find fruitful to make game designers more aware of such work, perhaps some ideas about a workshop/seminar.

Pervasive Expression: Workshop on Pervasive Visual, Auditory and Alternative Modality Information Display is a workshop Oorganized at the 6th International Conference on Pervasive Computing (Sunday 18 May 2008, Sydney, Australia).

It’s organized by Andrew Vande Moere, Kirsty Beilharz, Bert Bongers and Stephen Barrass and it will address the following topics:

“This workshop wants to bring together researchers, practitioners, technologists and artists from different domains, interested in the visual or auditory representation of information for users in the pervasive realm. We also hope to explore how novel visual, auditory and alternative modalities (e.g. tactile, olfactory, visceral) materials can function as a physical communicative layer that is truly pervasive. A few potential questions to be discussed in the workshop include:
- How to embed pervasive expressive displays in physical reality and materials, such as artifacts, garments and spaces?
- What are valid data mapping metaphors for expressive displays that are pervasive, and still can be intuitively understood?
- How can the design of pervasive expressive displays influence the experience (e.g. engagement, reflection, persuasion, interpretation), conviction, attitude or behavior of onlookers, users, wearers or any person in its vicinity?“

An interesting talk I attended yesterday at the Lyon Game Developers’ Conference was the on entitled “Creating Great Games for the Nintendo Wii and Its Unique Controller” by Philip and Andrew Oliver from Blitz Games:

“Even though the ideas are fairly simple to prototype, getting the feel of each control system just right was a challenge in itself. Detecting the correct motion of the controller had to be very specific and tested thoroughly as different people held the controllers in sometimes very different ways. The problem of how to convey the instructions for the controller types within the game had to be addressed. As we all know, nobody reads the manual, especially at the target age range. A series of images, text prompts and even animated movies were experimented with, all with different results. Testing and interpreting these methods were key to getting a successful title.“

Some of the results they described about designing wii movements:

“- swiping is tiring and it’s difficult to sync movements on the screen
- driving using the wiimote as a steering wheel that you can push/pull to accelerate/brake did not work because it was too tiring and small movements were unnoticed… and they noticed how an abstract notion like accelerating is better conveyed by pressing buttons
- people hold the wiimote VERY differently. Kids tend to hold it very loosely and do movements with large amplitudes whereas adults hold it more firmly and do not swing it away too much (as if it was a sacred grail or a TV remote control)
- certain controls that require player to hold the wiimote vertically did not work… because people do not hold it vertically: people do not look at the wiimote especially when standing. Asking someone to hold the wiimote vertically is difficult because it depends on situations: sit at a desk, lie on a bed, standing, standing with friends.
- distance between the wiimote and the screen was easily detected and that variable was very accurate to be employed in a game mechanic.“

Why do I blog this? these few elements are interesting and echoes with the (private) work I have done about this. More specially it resonates quite well with the importance of context in playing with the Wii. I am currently thinking about a study concerning how people have expectations about the wiimote, how they understand its usage. It would also be interesting to observe how people naturally hold the wiimote

Gave a talk at the European GDC today in Lyon. It was called “5 lessons about tangible user interfaces” and addressed an overview of classic misconceptions concerning tangible user interfaces. It’s actually a modified version of an earlier talk I gave last year at Nokia Design in Los Angeles; I added few things I’ve done since then. It’s always interesting to give such a talk to different audiences; the way it it is received by industrial designers is totally different by game developers, fully different insights and discussions. The slides are here (.pdf, 9.8Mb):

Tangible interfaces hold lots of promises ranging from being more intuitive or realistic, being more appealing to users to enabling people to get some physical exercises in the process. User experience research about it shows that things are not so simple. This presentation discusses 5 misconceptions and why they are wrong. Each can be exemplified by arguments drawn from user studies, which are of importance for game designers:
1. Inert objects do not lead to tangible interactions or how non-gestural interfaces such as TV remote control can be gestural
2. Direct mapping between the physical movement and the interaction in the digital world is simple and intuitive or how direct mapping is not always efficient for players or accurately detected.
3. physical interfaces offer a larger variety of control than standard controllers, and are more realistic and intuitive”: depending on the task, tangible interfaces actually do not necessarily lead to intuitiveness and ease of control.
4. The starting point of designing TUI is to look at real-life counterparts… so let’s design guns for shooting games, a flute for musical games…: there are actually other alternative that are almost never investigated, taking the opposite direction of direct mapping.
5. Tangible interfaces are ubiquitous and allows mobile/seamless interactions or how tangible interactions do not appear in a vacuum and lots of problems due to the context can happen.

The description of why these ideas are misconceptions lead to important implications and design lessons about how to go beyond current implementations in video games.

The conference was interesting, sometimes a bit too techno-centric for me, I will try to write something about my notes from the sessions I found relevant.

Preparing my talk for the Game Design Conference about tangible/gestural interfaces, I ran across this very interesting Wario Ware Walkthrough guide by G. Louie (not only curious because of its ASCII layout).

What struck me as very pertinent here is the nomenclature (the naming) and the description of moves. See for example this intriguing list:

“The Form Baton - The Balance Stone - The Remote Control - The Umbrella - The Handlebar - The Sketch Artist - The Chauffeur - The Samurai- The Tug-of-War -The Waiter - The Elephant - The Thumb Wrestler - The Discard -The Big Cheese - The Janitor - The Dumbbell - The Mohawk - The Finger Food - The Boxer -The Mortar and Pestle - The Diner“

With descriptions such as:

“ The Handlebar
“Turn the Form Baton sideways and grasp the ends firmly in both hands. Like riding a bicyle, perfecting this stance requires grace, steadiness, and tightshorts.”

To do The Handlebar, turn the Wii Remote so that the 2 button is on the right. Place hands over the top so that palms are on the buttons. Uses for this form would be pumping and balancing.
*Does not use remote sensor bar*“

Why do I blog this? what I find relevant here is the way people try to refer to gestures meant to control specific game interactions. Things get more complicated when the interface is gestural: how to name them? how to describe them not only to the users but in game guide? A solution as we se here is to rely on existing metaphors: type of activities (boxer), postures or animal that makes think of posture (elephant), moments (diner) or jobs (chauffeur).

In his blogpost about “unconscious gestures”, at a certain point, Matt Jones has a good rant about the cultural ownership on the touch interface of the iphone. As if all the other products which use touch/gestures had been copied (”with pride”):

“That last remark made me spit with anger - and I almost posted something very intemperate as a result. The work that all the teams within Nokia had put into developing touch UI got discounted, just like that, with a half-thought-through response in a press conference. I wish that huge software engineering outfits like S60 could move fast enough to ‘copy with pride’.

Sheesh.

Fact-of-the-matter is if you have roughly the same component pipeline, and you’re designing an interface used on-the-go by (human) fingers, you’re going to end up with a lot of the same UI principles.

But Apple executed first, and beautifully, and they win. They own it, culturally.“

Why do I blog this? speaking of cultural ownership, what is even more puzzling is all the press about the prominence of “minority report” in terms of interface paradigm.

As if every single gestural/touch UI that we have today have something to do with Minority Report, as if that movie taught people that it was where innovation in that field started. So you have newspaper article about the phone/table/display that-mimics-minority-report-gestural-interface. It’s really weird since the interface employed by Tom Cruise et al. are very different. There is really something here about the normative future created by a cultural artifact such as movies and tv series.

Apart from that, Matt’s also complains about the fact that what is pursued is more “deliberate touch interfaces - touch-as-manipulate-objects-on-screen rather than touch-as-manipulate-objects-in-the-world for now“, which is a relevant remark.

A follow-up to this blogpost, I started making a short list of “curious digital cameras”

Microsoft’s Sensecam, a wearable digital camera that is designed to take photographs passively, without user intervention, while it is being worn.

The Wingscapes BirdCam which uses an infrared sensor to detect birds and then automatically captures photos or videos (of your birds while you are away).

Tospom: a ball-shaped camera that takes pictures while playing catch.

Satugo: a bouncing ball camera.

Blinkcam takes a Polaroid picture everytime you blink.

Sascha Pohflepp’s blinks and buttons, a networked camera that capture a moment by continuously searching on the net for other photos that have been taken in the very same moment.

Spycamera stuff can also be found.

An HP camera that could be equipped with circuits that could be remotely triggered to blur the face of those who don’t want to have their photo taken.

Why do I blog this? it’s both the interest in automatic cameras as tools for User Experience research and as curious devices from the near future. Automatic camera can indeed be used to ask people to reflect on their activities (with some ethical limits) and weird cameras are very interesting devices to imagine new uses.

Please feel free to add any other references in the comment part of that blogpost, it’s good to keep track of stuff. There must be thousands of models and prototypes.

Folks from our lab today visited colleagues next door to see the current projects they’re working on, some sort of low-cost CAVE and haptic interfaces. Very instructive to try it out live. Quite big pieces of machinery anyway.

In the past twenty years, what were the improvements in haptics? What were the main lessons? What were successes an failures? Does the Wii count here?

Is that the future or the past of entertainment? Why?

So what if the Kärcher puzzi 400K was a video game interface?

Beyond traditional stick/cross paradigm, what about weird buttons/signs. As a design exercise, I would really enjoy seeing what one can do out of such interface. Take this as a design brief.

Why do I blog this? ideas for a course about design and foresight.

This transcript of an interview of Dan Saffer about his manifesto for gestural patterns for touch interaction is very pertinent. It’s mostly about this wiki resource which aims at collecting and disseminating gestural interface information and patterns, such as found on such devices as the iPhone and Wii (following a discussion Adaptive Path’s blog).

Some excerpts of this interview:

“How do you document this gesture where I’m sweeping my hand across the screen?” (…) This is our generation’s drag and drop.”
(…)
I felt it was a really important thing for interaction designers to be doing because, otherwise, we’re going to start to end up with a thousand different ways of turning on my TV where it’s like, “Is this the Microsoft TV where I have to snap my fingers three times or is it the Apple one where I twirl around in a circle?”
(…)
one of the nice things about having it be in a completely digital medium is that one of the problems with gestures is certainly documenting them. How do you describe something that’s not very ambiguous? It’s awfully difficult with words to describe gestures or even in diagrams to describe gestures.So having the ability to eventually put up movie clips showing this as a pattern with people moving their forefinger and thumb apart, for instance, having that kind of rich experience would be really nice on the website.“

(Picture taken from a wii game usability test I ran few months ago)

The examples he gives revolves around the Wii or the iPhone:

“The Wii certainly is very much about sort of movement in space. You’re not really touching anything except the controller. You’re kind of indirectly using a gesture. With the touch screen on the iPhone and other things, your fingertip is actually touching the device that you’re manipulating. So there is this gradation there.“

Why do I blog this? this is indeed an interesting issue, how you describe these movements? can we have a grammar (i.e. a set of patterns). This has some tight connections with a project I am involved in that tries to map the wiimote and nunchuk movements of existing games in a database, this will then allow to analyze them and document their relative importance.

There is an article in Technology Review about tangible interfaces for video-games by Erica Naone. It’s basically about a vest (called 3rd Space) that aims at bringing more realism to the game experience by simulating impacts. It’s based on pneumatic cells which produce impacts of various strength in different locations on the player’s torso.

The article gives a brief overview of user experience issues:

“Force feedback devices are already popular among gamers, and Ombrellaro says that his vest promises an even more realistic experience than today’s vibrating controllers. “The drama moment with this is getting shot in the back in a first-person game,” he says. In market tests for the vest, he says, people would turn around in surprise when they felt the impact in the back, even though they knew intellectually to expect it. Based on feedback from its tests, the company chose a standard strength of impact, which is palpable but not bruising. “We’re pushing the edge,” he says. “We’re still keeping it very fun but, at the same time, giving you tactile cues that are important. There’s even subtly a message–that there are consequences to shooting people.” Ombrellaro says that he also plans to ship vests with a more powerful compressor for a subset of gamers who want to feel stronger impacts and for use in military and police training.”

Why do I blog this? video-games (as well as lots of digital environments) engage people in immersive experience but the body is often less involved (although the Wii suffers less from that issue…). In this case, even though the player cannot be hurt, the proprioceptive sense is mobilized in an interesting way.

“Experiencing ‘Touch’ in Mobile Mixed Reality Games” by Paul Coulton, Omer Rashid and Will Bamford is one of these papers which stand on my desktop for ages, waiting to be parsed and analyze (among lots of others). Found time to read it today in the train (heading back to Geneva from a three-day meeting series in Paris), possibly caused by a meeting with Rafi Haladjian at Violet yesterday.

The paper describes the user experience of mobile phones equipped with RFID/NFC to play different games that involves RFID-tagged objects. NFC stands for “Near Field Communications” and is an interface and protocol built on top of RFID. The games described in this paper are PAC-LAN (a Pacman-like game in physical space), Mobspray (a virtual graffiti system) and MobHunt (a treasure hunt game).

The most interesting part of the paper (wrt my research) concerns the results from the results. They found that the system usability (touching tags) was efficient and not prone to social acceptability issue. Excerpts from the results:

The users found the objects very useful compared with just placing an RFID tag at a location as they found it much easier to see and felt it added to the immersion within the game play. (…) Another aspect of the objects was that for PAC-LAN, which was played at a much faster pace than the other two games, the players felt that the game disks were an important element of the game experience and minimized the time they had to spend checking their position on the mobile phone screen. Having played many location based games that rely on purely virtual objects we observed that players often become completely focused on the screen to guide them and often become oblivious to their environment which both defeats the premise of mixed reality gaming and can also be very dangerous.
(…)
One of the other aspects we experimented with was related to giving the user feedback after they have successfully read or
written from or to a tag. For PAC-LAN we initially created version that had either visual feedback, through a pop-up note, or
audio feedback, by playing a short tune. The audio feedback was unanimously preferred as players were often running at speed and the audio feedback was perceived much less intrusive on the game and harder to miss

Why do I blog this? after a discussion yesterday about gaming, RFID and social computing, it was funny to get back to this paper. Some curious things to draw here about feedback and immersion, quite important factor when designing gaming systems.

Coulton, P., Rashid, O., and Bamford, W., “Experiencing ‘Touch’ in Mobile Mixed Reality Games”, Proceedings of The Fourth Annual International Conference in Computer Game Design and Technology, Liverpool, 15th – 16th November 2006, pp 68-7

Read “Questioning Ubiquitous Computing” by Araya this morning on the train. Although the paper dates from 1995, it’s still highly relevant considering how it gives a critical analysis of the technological proposals of ubicomp. The author aimed at criticizing the “technical thining”, i.e. the kind of assumptions, justifications and modes of reasoning that underlies Ubiquitous Computing. It’s important to keep in mind though that what the author judges here is rather the description of Ubicomp based on Mark Weiser’s papers and less the concrete instantiations that has been designed afterwards.

Araya’s claim is that ubicomp leads to “displacement, transformation, substitution, or loss of fundamental properties of aspects of the “world” in such a way that its otherness is increasingly eliminated“. The world becomes then “a subservient artifact”. Some excerpts that I found interesting:

“What is striking about most of these scenarios is the marginal and irrelevant character of the needs referred to in them and of the envisioned enhancements of the activities (e.g., elevators stop at the right floor, rooms greet people by name, secretaries instantly know the location of employees). Although it is tempting to discard this marginality as if it were only an impression produced by the chosen scenarios, we believe that it has a more fundamental character.
(…)
Even more striking is the stark contrast between the marginality of the enhancements and the complexity of the computing infrastructure required to achieve them. (…) The question then becomes, if not driven by the purpose of satisfying significant human needs how does Ubiquitous Computing justify itself?“

His answer to this question is: technology. Although he acknowledges that human need may have been historically generated sometimes by technologies, Araya points to the fact that the scale and scope of new needs to be satisfied by ubicomp are unprecedented. He then worries about this technological absolutism in which the technological thinking is never called into question (”the primacy of the unfolding of technology over the satisfaction of humans needs, and the self-sufficiency of this unfolding are taken as absolute givens. “). Down the road, this leads to a situation in which technology does not require any justification outside of itself.

Why do I blog this? although a bit left-over (I haven’t seen lots of citations) it’s refreshing to run across these critiques. Especially, the discussion about the gap between so-called needs and the infrastructures to be put into place to meet them.

One of the example he takes is very close to things I’m interested in, namely the representation of physical space through digital means:

“By disseminating digital surrogates of the world, that is, digital representations of partial aspects of the world which have been subject to more or less intense pre-processing. As the following scenario illustrates, the utility of these surrogates is not confined to office or working situations, but could also have certain uses at home: “Sal looks…at her neighborhood…through…electronic trails that have been kept for her of neighbors coming and going through the early morning… Time markers and electronic tracks on the neighborhood map let Sal feel cozy in her street
(…)
What does the scenario in which “Sal looks at electronic tracks of neighbors coming and going in the morning” tell us? The “need for social interaction” has been anticipated in the responsive environment and elaborate surrogates of relevant aspects of the world have been prepared. The street, the morning, the neighbors and their encounters have been displaced in time and space and replaced by surrogates, suffering a deep transformation in the process. Entire aspects of the situation have been filtered away and they can no longer surprise us. The electronic surrogates of the street situation live now in a different world, a world in which surrogates of the past can be replayed at any time, replicated, and distributed at will.“

Araya, A. A. (1995). Questioning Ubiquitous Computing. In ACM Conference on Computer Science, pages 230–237.

Concrete is a quite trendy store in Amsterdam, NL that sells clothes, toyz as well as designers’ accessories. Yesterday, it proposed some work by Casey Reas (viz/image design) and Cait Reas (dresses).

Why do I blog this? what was intriguing there was the presence of the book “Processing: A Programming Handbook for Visual Designers and Artists” (Casey Reas, Ben Fry) that is, essentially, the bible to employ this programming language, which allowed the design of the viz on the wall and the dress such as the one represented on the picture above. Physical-to-digital-to-physical translation.