In “Technology paternalism – wider implications of ubiquitous computing”, S. Spiekermann and F. Pallas deal with how people can maintain control in environments that are supposed to be totally automated. They coined the term “technology paternalism” to describe the situation where “people may be subdued to machines’ autonomous actions“. They take the example of car that beeps when you don’t fasten your seatblet and show how such situations meet the same criteria as the one that define paternalism:

“ the definition of Technology Paternalism extends the general notion of paternalism with respect to two aspects: one is that actions are being taken autonomously by machines. The other one is that by their coded rules, machines can become ‘absolute’ forces and therefore may not be overrulable any more.“

And discuss that in conjunction with Weiser’s notion of calm computing:

“If machines are controlled, then they are not calm any more. There is a clear disaccord between the concept of disappearing technologies and the attempt to remain in control. Control premises attention and visibility whilst Ubicomp environments are designed to be invisible and seamlessly adaptive. Can this dissonance really ever be resolved? “

And of course there is a part about who’s responsible of technology paternalism:

“Of course, this power does not lie in the hands of technology itself. Technology only follows rules implemented into it. Therefore, the question arises: who WILL be the real patrons behind Technology Paternalism if it were to become a reality? Who will decide about the rules, the ‘rights’ and ‘wrongs’ of every-day actions?
(…)
three groups as the potential patrons behind Technology Paternalism: engineers and marketers of Ubicomp technologies as well as regulators influencing application design. “

Why do I blog this? Some relevant issues regarding the notion of control in ubicomp. The authors finally come up with a series of recommendations. The one that strikes me as fundamental is the following: “there should be a general possibility to overrule ‘decisions’ made by technology and any exceptions from this should be considered very carefully“. The notion of exception is a crux issue that is often diminished by lots of engineers I talked to wrt to autonomous technologies such as “intelligent fridges” or location-based services. Exceptions breaks patterns and habits tracked by sensors, disrupt machine learning algorithm and are eventually impediments to prediction-based system that would send emergency messages to 911 because granny did not open her fridge for 2 weeks (because she unexpectedly decided to visit her grandson).

In the last issue of ACM interactions, there is an interesting paper called “When users “do” the Ubicomp” by Antti Oulasvirta. The paper starts off by discussing the 2 perspectives of ubiquitous computing: on the one hand, the sort of research/outcomes you see in conferences and on the other hand, what the author calls:

“present-day IT infrastructure, the real ubicomp, is a massive noncentralized agglomeration of the devices, connectivity and electricity means, applications, services, and interfaces, as well as material objects such as cables and meeting rooms and support surfaces that have emerged almost anarchistically, without a recognized set of guiding principles“

… which is very close to what Bell and Dourish described in their paper Yesterday’s tomorrows: notes on ubiquitous computing’s dominant vision. Oulasvirta wonders if the inherent complexity of ubicomp is one one key explanation to why these applications have not conquered the consumer market. Relying on different results from ubicomp studies, he presents ” a laundry list of approaches to improving ubicomp infrastructures”:

“1) minimizing overheads that create temporal seams between activities;
2) making remote but important resources, such as connectivity or cables, better transparent locally and digitally;
3) propagating metadata on migration of data from device to device;
4) supporting ad hoc uses of proximate devices’ resources like projectors, keyboards, and displays;
5) triggering digital events like synchronization of predetermined documents with physical gestures;
6) supporting appropriation of material properties for support surfaces“

Why do I blog this? some interesting issues to consider here, although I am not sure (or perhaps interested) in everyone of them, some are close to current interests (for example (2)).

What does handheld electronic game such as Parker Brother’s Split Second can teach us?

The curious box/enclosure? the super straight select-start-4-arrows buttons? the rockin’ analogic screen? the okay-for left-handed and okay for right-handed design? the symmetry of the system with sound on one part and display on the other?

Maybe it’s the whole experience or even the expectations back in the days, when playing with very near 3×15 red matrix was like being immersed in Tron. What impresses me now is the “one-device = one game/purpose” equation. The interface was so rough and basic that you could only play one sort of game. It was even crazier with Nintendo handheld a la Donkey Kong since part of the level design was DRAWN and PRINTED on the screen. Would there still be devices like this? Or only converged phones?

Very curiously, this sort of electronic games have always received a very low interest from both thinkers and academics. In the book “Electronic Plastic (see also here), the authors of that nice compendium state how “the recourse to supposedly primitive games leads us back to the creative source of the contemporary entertainment revolution” and that these game provide as much fun as recent Sony or Nintendo platforms.

The near future laboratory (Fabien Girardin and Julian Bleecker and myself) organize a workshop during LIFT08 about the failures of ubiquitous computing.

The workshop is called “Ubiquitous computing: visions, failures and new interaction rituals“:

“We propose to look backward and discusses why we have not reached what has been described in the last 5 years of ubiquitous computing. How might we criticize assumptions and build upon existing models and approaches to design in this context? Can we learn from the discrepancy between the utopia of ubicomp and its deployed reality?

The purpose is to generate debate about the design and integration of ubiquitous systems based on case studies proposed from workshop participants. Moreover, we want to open up a debate around the future of those systems as well as the adoption by a large user base.

The session will start by a short presentation by participants who will each have to describe in 2-3 minutes a ubiquitous computing system that failed and give reasons or causes for that.
…“

Why do I blog this? LIFT is always an enjoyable moments for workshops, let’s see what emerge out of that one.

Currently in Aix-en-Provence, at the School of Art where I’ve given a talk yesterday about tangible interfaces (a rerun from my GDC2007 presentation). The talk was part of a workshop called “Workshop Wiimote Hacking. The whole thing was about how to hack the Nintendo Wiimote and turn it into a tool that artists can employ. Students are involved in the process of adding new sensors as well as defining new sorts of usage. Thanks France Cadet for the invitation and Douglas for having taken time to discuss.

The discussion after the presentation revolved around:
- How the wiimote might have the potential to become a sort of standard in the living room as the controller. I personally don’t predict anything about this. However from current observations of practices, I do think that the Wii is much more than video game device and play an interesting role of multimedia platform in the living room (with photos, usage of personalized mii). In addition, the presence of very focused applications such as wii questions turns it into a platform where facebook-like small application can be played by the family.
- Why the game design ideas we have so far on the Wii are so conservative… which turned into why the video game industry is so conservative or how the whole economic system is so controlled by the marketing crowd that it’s difficult to go beyond usage of the wiimote as sword/steering-wheel/magic wand. This is a sort of over-statement but it’s actually close to the reality.
- How this work about tangible interfaces relate to my other work about mobile and pervasive gaming? we discussed the notion of granularity and how complex the game system is when the dimensions reach the city level. Much more than just gesturing in one’s own living room, using the city as a game field is complex for lots of reasons (technical, infrastructural, difficulty to have a continuous experience), etc.

Fortunately, there was also an exhibit about games as medium for artists to create and tweak digital worlds. Called gamerz02, there was a bunch of very curious projects.

The one that attracted my attention was Patch&KO (Antonin Fourneau and Manuel Braun): instead of using a joystick people can play Street Fighter 2 using a pachinko interface. In a sense, the player’s ability to control the character is disrupted by the semi-random movement of the metal balls. As the designers state, the player has to accept a loss of control. Slightly related was this Tictactoe played by robotic arms I blogged about the other day. In this case, what was explored was that the Tic Tac Toe is a curious game in which the only way to win is to rely on the opponent’s fatigue and loose of sight in the game. In the context of two mechanical arms played by a computer, the surest way to win is to avoid playing. Finally, I was also interested by Tchouri by Pascal Silondi, a sort of knife-based interface much more intriguing than always-seen magic wands. Some of the pieces there were maybe slightly easy-going and naive but the whole things made sense and I found find fruitful to make game designers more aware of such work, perhaps some ideas about a workshop/seminar.

Pervasive Expression: Workshop on Pervasive Visual, Auditory and Alternative Modality Information Display is a workshop Oorganized at the 6th International Conference on Pervasive Computing (Sunday 18 May 2008, Sydney, Australia).

It’s organized by Andrew Vande Moere, Kirsty Beilharz, Bert Bongers and Stephen Barrass and it will address the following topics:

“This workshop wants to bring together researchers, practitioners, technologists and artists from different domains, interested in the visual or auditory representation of information for users in the pervasive realm. We also hope to explore how novel visual, auditory and alternative modalities (e.g. tactile, olfactory, visceral) materials can function as a physical communicative layer that is truly pervasive. A few potential questions to be discussed in the workshop include:
- How to embed pervasive expressive displays in physical reality and materials, such as artifacts, garments and spaces?
- What are valid data mapping metaphors for expressive displays that are pervasive, and still can be intuitively understood?
- How can the design of pervasive expressive displays influence the experience (e.g. engagement, reflection, persuasion, interpretation), conviction, attitude or behavior of onlookers, users, wearers or any person in its vicinity?“

An interesting talk I attended yesterday at the Lyon Game Developers’ Conference was the on entitled “Creating Great Games for the Nintendo Wii and Its Unique Controller” by Philip and Andrew Oliver from Blitz Games:

“Even though the ideas are fairly simple to prototype, getting the feel of each control system just right was a challenge in itself. Detecting the correct motion of the controller had to be very specific and tested thoroughly as different people held the controllers in sometimes very different ways. The problem of how to convey the instructions for the controller types within the game had to be addressed. As we all know, nobody reads the manual, especially at the target age range. A series of images, text prompts and even animated movies were experimented with, all with different results. Testing and interpreting these methods were key to getting a successful title.“

Some of the results they described about designing wii movements:

“- swiping is tiring and it’s difficult to sync movements on the screen
- driving using the wiimote as a steering wheel that you can push/pull to accelerate/brake did not work because it was too tiring and small movements were unnoticed… and they noticed how an abstract notion like accelerating is better conveyed by pressing buttons
- people hold the wiimote VERY differently. Kids tend to hold it very loosely and do movements with large amplitudes whereas adults hold it more firmly and do not swing it away too much (as if it was a sacred grail or a TV remote control)
- certain controls that require player to hold the wiimote vertically did not work… because people do not hold it vertically: people do not look at the wiimote especially when standing. Asking someone to hold the wiimote vertically is difficult because it depends on situations: sit at a desk, lie on a bed, standing, standing with friends.
- distance between the wiimote and the screen was easily detected and that variable was very accurate to be employed in a game mechanic.“

Why do I blog this? these few elements are interesting and echoes with the (private) work I have done about this. More specially it resonates quite well with the importance of context in playing with the Wii. I am currently thinking about a study concerning how people have expectations about the wiimote, how they understand its usage. It would also be interesting to observe how people naturally hold the wiimote

Gave a talk at the European GDC today in Lyon. It was called “5 lessons about tangible user interfaces” and addressed an overview of classic misconceptions concerning tangible user interfaces. It’s actually a modified version of an earlier talk I gave last year at Nokia Design in Los Angeles; I added few things I’ve done since then. It’s always interesting to give such a talk to different audiences; the way it it is received by industrial designers is totally different by game developers, fully different insights and discussions. The slides are here (.pdf, 9.8Mb):

Tangible interfaces hold lots of promises ranging from being more intuitive or realistic, being more appealing to users to enabling people to get some physical exercises in the process. User experience research about it shows that things are not so simple. This presentation discusses 5 misconceptions and why they are wrong. Each can be exemplified by arguments drawn from user studies, which are of importance for game designers:
1. Inert objects do not lead to tangible interactions or how non-gestural interfaces such as TV remote control can be gestural
2. Direct mapping between the physical movement and the interaction in the digital world is simple and intuitive or how direct mapping is not always efficient for players or accurately detected.
3. physical interfaces offer a larger variety of control than standard controllers, and are more realistic and intuitive”: depending on the task, tangible interfaces actually do not necessarily lead to intuitiveness and ease of control.
4. The starting point of designing TUI is to look at real-life counterparts… so let’s design guns for shooting games, a flute for musical games…: there are actually other alternative that are almost never investigated, taking the opposite direction of direct mapping.
5. Tangible interfaces are ubiquitous and allows mobile/seamless interactions or how tangible interactions do not appear in a vacuum and lots of problems due to the context can happen.

The description of why these ideas are misconceptions lead to important implications and design lessons about how to go beyond current implementations in video games.

The conference was interesting, sometimes a bit too techno-centric for me, I will try to write something about my notes from the sessions I found relevant.

Preparing my talk for the Game Design Conference about tangible/gestural interfaces, I ran across this very interesting Wario Ware Walkthrough guide by G. Louie (not only curious because of its ASCII layout).

What struck me as very pertinent here is the nomenclature (the naming) and the description of moves. See for example this intriguing list:

“The Form Baton - The Balance Stone - The Remote Control - The Umbrella - The Handlebar - The Sketch Artist - The Chauffeur - The Samurai- The Tug-of-War -The Waiter - The Elephant - The Thumb Wrestler - The Discard -The Big Cheese - The Janitor - The Dumbbell - The Mohawk - The Finger Food - The Boxer -The Mortar and Pestle - The Diner“

With descriptions such as:

“ The Handlebar
“Turn the Form Baton sideways and grasp the ends firmly in both hands. Like riding a bicyle, perfecting this stance requires grace, steadiness, and tightshorts.”

To do The Handlebar, turn the Wii Remote so that the 2 button is on the right. Place hands over the top so that palms are on the buttons. Uses for this form would be pumping and balancing.
*Does not use remote sensor bar*“

Why do I blog this? what I find relevant here is the way people try to refer to gestures meant to control specific game interactions. Things get more complicated when the interface is gestural: how to name them? how to describe them not only to the users but in game guide? A solution as we se here is to rely on existing metaphors: type of activities (boxer), postures or animal that makes think of posture (elephant), moments (diner) or jobs (chauffeur).

In his blogpost about “unconscious gestures”, at a certain point, Matt Jones has a good rant about the cultural ownership on the touch interface of the iphone. As if all the other products which use touch/gestures had been copied (”with pride”):

“That last remark made me spit with anger - and I almost posted something very intemperate as a result. The work that all the teams within Nokia had put into developing touch UI got discounted, just like that, with a half-thought-through response in a press conference. I wish that huge software engineering outfits like S60 could move fast enough to ‘copy with pride’.

Sheesh.

Fact-of-the-matter is if you have roughly the same component pipeline, and you’re designing an interface used on-the-go by (human) fingers, you’re going to end up with a lot of the same UI principles.

But Apple executed first, and beautifully, and they win. They own it, culturally.“

Why do I blog this? speaking of cultural ownership, what is even more puzzling is all the press about the prominence of “minority report” in terms of interface paradigm.

As if every single gestural/touch UI that we have today have something to do with Minority Report, as if that movie taught people that it was where innovation in that field started. So you have newspaper article about the phone/table/display that-mimics-minority-report-gestural-interface. It’s really weird since the interface employed by Tom Cruise et al. are very different. There is really something here about the normative future created by a cultural artifact such as movies and tv series.

Apart from that, Matt’s also complains about the fact that what is pursued is more “deliberate touch interfaces - touch-as-manipulate-objects-on-screen rather than touch-as-manipulate-objects-in-the-world for now“, which is a relevant remark.

A follow-up to this blogpost, I started making a short list of “curious digital cameras”

Microsoft’s Sensecam, a wearable digital camera that is designed to take photographs passively, without user intervention, while it is being worn.

The Wingscapes BirdCam which uses an infrared sensor to detect birds and then automatically captures photos or videos (of your birds while you are away).

Tospom: a ball-shaped camera that takes pictures while playing catch.

Satugo: a bouncing ball camera.

Blinkcam takes a Polaroid picture everytime you blink.

Sascha Pohflepp’s blinks and buttons, a networked camera that capture a moment by continuously searching on the net for other photos that have been taken in the very same moment.

Spycamera stuff can also be found.

An HP camera that could be equipped with circuits that could be remotely triggered to blur the face of those who don’t want to have their photo taken.

Why do I blog this? it’s both the interest in automatic cameras as tools for User Experience research and as curious devices from the near future. Automatic camera can indeed be used to ask people to reflect on their activities (with some ethical limits) and weird cameras are very interesting devices to imagine new uses.

Please feel free to add any other references in the comment part of that blogpost, it’s good to keep track of stuff. There must be thousands of models and prototypes.

Folks from our lab today visited colleagues next door to see the current projects they’re working on, some sort of low-cost CAVE and haptic interfaces. Very instructive to try it out live. Quite big pieces of machinery anyway.

In the past twenty years, what were the improvements in haptics? What were the main lessons? What were successes an failures? Does the Wii count here?

Is that the future or the past of entertainment? Why?

So what if the Kärcher puzzi 400K was a video game interface?

Beyond traditional stick/cross paradigm, what about weird buttons/signs. As a design exercise, I would really enjoy seeing what one can do out of such interface. Take this as a design brief.

Why do I blog this? ideas for a course about design and foresight.

This transcript of an interview of Dan Saffer about his manifesto for gestural patterns for touch interaction is very pertinent. It’s mostly about this wiki resource which aims at collecting and disseminating gestural interface information and patterns, such as found on such devices as the iPhone and Wii (following a discussion Adaptive Path’s blog).

Some excerpts of this interview:

“How do you document this gesture where I’m sweeping my hand across the screen?” (…) This is our generation’s drag and drop.”
(…)
I felt it was a really important thing for interaction designers to be doing because, otherwise, we’re going to start to end up with a thousand different ways of turning on my TV where it’s like, “Is this the Microsoft TV where I have to snap my fingers three times or is it the Apple one where I twirl around in a circle?”
(…)
one of the nice things about having it be in a completely digital medium is that one of the problems with gestures is certainly documenting them. How do you describe something that’s not very ambiguous? It’s awfully difficult with words to describe gestures or even in diagrams to describe gestures.So having the ability to eventually put up movie clips showing this as a pattern with people moving their forefinger and thumb apart, for instance, having that kind of rich experience would be really nice on the website.“

(Picture taken from a wii game usability test I ran few months ago)

The examples he gives revolves around the Wii or the iPhone:

“The Wii certainly is very much about sort of movement in space. You’re not really touching anything except the controller. You’re kind of indirectly using a gesture. With the touch screen on the iPhone and other things, your fingertip is actually touching the device that you’re manipulating. So there is this gradation there.“

Why do I blog this? this is indeed an interesting issue, how you describe these movements? can we have a grammar (i.e. a set of patterns). This has some tight connections with a project I am involved in that tries to map the wiimote and nunchuk movements of existing games in a database, this will then allow to analyze them and document their relative importance.

There is an article in Technology Review about tangible interfaces for video-games by Erica Naone. It’s basically about a vest (called 3rd Space) that aims at bringing more realism to the game experience by simulating impacts. It’s based on pneumatic cells which produce impacts of various strength in different locations on the player’s torso.

The article gives a brief overview of user experience issues:

“Force feedback devices are already popular among gamers, and Ombrellaro says that his vest promises an even more realistic experience than today’s vibrating controllers. “The drama moment with this is getting shot in the back in a first-person game,” he says. In market tests for the vest, he says, people would turn around in surprise when they felt the impact in the back, even though they knew intellectually to expect it. Based on feedback from its tests, the company chose a standard strength of impact, which is palpable but not bruising. “We’re pushing the edge,” he says. “We’re still keeping it very fun but, at the same time, giving you tactile cues that are important. There’s even subtly a message–that there are consequences to shooting people.” Ombrellaro says that he also plans to ship vests with a more powerful compressor for a subset of gamers who want to feel stronger impacts and for use in military and police training.”

Why do I blog this? video-games (as well as lots of digital environments) engage people in immersive experience but the body is often less involved (although the Wii suffers less from that issue…). In this case, even though the player cannot be hurt, the proprioceptive sense is mobilized in an interesting way.