My paper for the CHI 2007 doctoral consortium has been accepted.

Bridging the social-technical gap in location-aware computing

Abstract. Building ubiquitous applications that exploit location requires integrating underlying infrastructure for linking sensors with high-level representation of the measure space to support human activities. However, the real-world constraints limit the efficiency of location technologies. The inherent spatial uncertainty embedded in mobile and location systems constantly challenges the coexistence of digital and physical spaces. Consequently, the technical mechanisms fail to match the highly flexible, nuanced, and contextual human spatial activities. These discrepancies generate a social-technical gap between what should be socially supported and what can be technically achieved. My research aims at exploring, and hopefully reducing this gap in the context of location-aware computing.

Relation to my thesis: “In total, we received 96 submissions and 15 were accepted to participate at the conference”. This is a clear sign that the definition and expression of my ideas have matured. CHI 2007 will take place in San Jose, CA between April 28 and May 3.

Davies, N., Landay, J., Hudson, S., and Schmidt, A. Guest editors’ introduction: Rapid prototyping for ubiquitous computing. Pervasive Computing, IEEE 4, 4 (Oct.-Dec. 2005), 15– 17.

The authors suggest that, as in earlier HCI efforts, the progress in prototyping methods and tools is central overcome the barriers to widespread development and deployment of ubicomp. Rapid prototyping techniques can partially solve the current dilemma of researchers and developers who must concentrate on their specific area to advance technology rather than expend effort on broad system-implementation issues. Low-fidelity prototyping can adapt to the pervasive computing requirements and proves to be an essential means to address the questions about user performance and user acceptance.

This paper serves as introduction to Prototypes in the Wild: Lessons from Three Ubicomp Systems.

Relation to my thesis: A useful ref for my “in sitiu” approach of location-aware system evaluation. Actually, the authors describe it as:

Research shows that prototyping and deploying systems for study is important to understanding how systems fit into the user’s world and how they can be used effectively. Designing, building, and deploying systems help both researchers and developers better understand a particular application domain’s key issues. This issue provides a rich body of experience in issues associated with prototype deployment.

Rogers, Y. Moving on from weiser’s vision of calm computing: Engaging ubicomp experiences. In Ubicomp (2006), pp. 404–421.

This paper urges for an alternative agenda in ubicomp research that shifts from Weiser’s calm vision to engaging people (i.e. proactive computing, persuasive computing, engaged living). Yvonne Rogers acknowledges that research in context awareness, ambient intelligence and monitoring/tracking have been somehow fruitful. However they have yet failed to reach Weiser’s world. Indeed, there is an enormous gap between the dream of conformable, informed and effortless living and the accomplishment of UbiComp research. In fact, the fundamental stumbling block has been harnessing the huge variability in what people do, their motives for doing it, when they do it and how they do it. While it has been possible to develop a range of simple ubicomp systems that can offer relevant information at opportune moment, it is proving to be much more difficult to build truly smart systems that can understand or accurately model people’s behaviors, moods and intentions. This makes it difficult, if not impossible, to try to implement context in any practical sense and from which to make sensible predictions about what someone is feeling, wanting or needing at a given moment. Therefore, ubicomp technologies should be designed not to do things for people but to engage them more actively in what they currently do. Rather than calm living it promotes engaged living, where technology is designed to enable people to do what they want, need or never even considered before by acting in and upon the environment. Examples include extending and supporting personal, cognitive and social processes such as habit-changing, problem solving, creating, analyzing, learning or performing a skill.

The author mentions the problems of calm computing in the most prominent ubicomp research themes (i.e. context-aware computing, ambient/ubiquitous intelligence and recording/tracking and monitoring).

Context-awareness
Key questions in context-aware computing concern what to sense, what form and what kind of information to represent to augment ongoing activities. Many of the sensor technologies, however, have been beset with detection and precision limitations, sometimes resulting in unreliable and inaccurate data. While newer technological developments may enable more accurate data to be detected and collected it. However, people often behave in unpredictable and subtle ways in their day-to-day contexts. Therefore, it is likely that context-aware systems will only ever be successful in highly constrained settings.

Ambient and Ubiquitous Intelligence
While there have been significant advances in computer vision, speech recognition and gesture-based detection, the reality of multimodal interfaces – that can predict and deliver with accuracy and sensitivity what is assumed people want or need – is a long way off. In consequence, when a ubiquitous computing system gets it wrong – which is likely to be considerably more frequent – it is likely to be more frustrating and we are likely to be less forgiving.

Recording, Tracking and Monitoring
Much of the discussion about the human aspects in ubicomp has been primarily about the trade-offs between security and privacy, convenience and privacy, and informedness and privacy. This focus has often been at the expense of other human concerns receiving less airing, such as how recording, tracking and re-representing movements and other information can be used to facilitate social and cognitive processes.

Yvonne mentions 2 goals of my research, one being to use ubicomp technologies in the wild, the other to evaluate how to present data and information:

In addition, more studies are needed of UbiComp technologies being used in situ or the wild – to help illuminate how people can construct, appropriate and use them. With respect to interaction design issues, we need to consider how to represent and present data and information that will enable people to more extensively compute, analyze, integrate, inquire and make decisions; how to design appropriate kinds of interfaces and interaction styles for combinations of devices, displays and tools; and how to provide transparent systems that people can understand sufficiently to know how to control and interact with them.

Currently, the more engaging approach is beginning to happen through the areas of playful and learning practices, scientific practices and persuasive practices.

As already mentioned in Comparing AI’s Failures with Ubicomp’s Visions, Yvonne Rogers concludes on “strong” and “weak” UbiComp.

Just as ‘strong’ AI failed to achieve its goals – where it was assumed that “the computer is not merely a tool in the study of the mind; rather, the appropriately programmed computer really is a mind”, it appears that ‘strong’ UbiComp is suffering from the same fate. And just as ‘weak’ AI2 revived AI’s fortunes, so, too, can ‘weak’ UbiComp bring success to the field.

Relation to my thesis: I would argue that current “strong” UbiComp problems not only lays on modelling people and their activities, but also in the integration ubicomp systems in the real-world (e.g. co-existence of systems, real-world constraints). I enjoy the difference between what is “relevant” and what is “smart”, as I find the word smart or intelligent are widely (over)misused. Finally, the agenda proposed in this article, goes in the direction of my research: in sitiu (out of the lab) studies, investigate the playful approach of ubicomp and how to present relevant information rather than seeking the seamlessness utopia.

Suomela, R., and Lehikoinen, J. Taxonomy for visualizing location-based information. Virtual Reality 8, 2 (2004), 71–82.

This paper concentrates on analyzing different visualizations for location-based applications. It studies two factors that affect the visualization of location-based data. The two factors are the environment model they use, ranging from three dimensions (3D) to no dimensions (0D) at all; and the viewpoint, whether it is a first-person or a third-person view. The authors suggest a taxonomy featuring model-views (MV) for visualizing location-based data.

The environment model is used to denote how many dimensions the application uses in visualizing the environment. If no environment model is used, the user does not gain specific location information of an object, except that the object might be somewhere close by.

• 3D environment model: these applications have an accurate 3D model of the environment and they place the location-based data onto its actual location in either the virtual or augmented view
• 2D environment model: the locations of the virtual objects are accurately projected onto a plane
• 1D environment model: application only shows one aspect of the location-based data
• No environment model: the applications present the data to the user but nothing about its location or relation to the user

The user’s view to the location-based data is one of the two:

• First person view: the user views the location-based data from a user-centric view, and the location-based data is spread around him or her
• Third person view: the user views both the location-based data and his or her representation

The first-person views, MV(x, 1), can help the user in wayfinding and provide additional information on objects. It is easy to show where the next waypoint is or the direction to it, and all visible real-world objects can be digitally augmented with additional information. The third-person views on the other hand can show the user a much wider area in all the directions around the user, as they are not restricted to the user’s current viewpoint and orientation.

Navigational tasks and views
For some tasks, the egocentric views are better, while for other tasks, some other views would be preferred. Navigational tasks with digital maps can be defined as searching tasks (naïve search and primed search), and exploration tasks. A fourth task can be defined as a targeted search. In a targeted search, the target is shown on the map; in primed search, the target is known, but does not appear on the map; in naïve search, there is no a prior knowledge of the position of the target, and the target is not shown on the map; in exploration, no specific target has been set.

Alignment
An important aspect concerning maps and navigation is alignment; that is, it specifies how the map is oriented with respect to the user and the environment. A map may be reader aligned, in which case the orientation of the map remains constant with regard to the reader’s body. An environment-aligned map, on the contrary, is oriented consistently with regard to the environment; in other words, north on the map always corresponds to north in the environment.

People difficulties in map reading
Even though a 2D map display is a well known visualization technique, it has been found that the most severe problem with
using traditional 2D maps is the inability to understand the spatial relationships between the real-world objects, and, therefore, to match the map and terrain model in one’s mind (a study shows suggests that up to 64% of the population have difficulties in map reading.

Models and location accuracy
Not all of the models need similar accuracy for the location. The AR applications need to determine the user’s viewpoint very accurately, as they need to know how the real world is aligned to the user. On the other hand, applications that only list the virtual objects do
not need to know the location very accurately.

Examples from the taxonomy

3D environment model: first person view; MV(3, 1)

3D environment model: third person view; MV(3, 3)

2D environment model: first person view; MV(2, 1)

For example, car navigation system
2D environment model: third person view; MV(2, 3)

The application only needs to know the user’s location; other sensor information is not necessary. Increasing the map scale can compensate for an inaccurate location of the user, but if the user’s location is not know accurately, there is no point in showing “You Are Here”. Previous studies have shown that a map is easier to use if it is aligned forward up.
1D environment model: third person view; MV(1, 3)

Relation to my thesis: The authors mentions the relation between the model and accuracy to position the user’s viewpoint. Yet, they suggest that virtual objects do not need to be perfectly located. If the hardware does not have accurate sensors, the third-person views might be more user-friendly. This still has to be studies and proved. Moreover, they mention that “location-based information is, typically, a set of virtual objects in a certain area, and that virtual object have a precise location in the real world”. I do not agree about virtual object having a precise location, when one think for example of Flickr geotagged images attached to the an area (i.e. a place) and not a position. Many times, an area does not have clear limits such as walls and people have different perspective of an area.
The model lack of the time dimension, since virtual objects are not necessarily fixed.

Relevant reference:
Aretz AJ (1991) The design of electronic map displays. Hum Factors 33(1):85–101

Borriello, G., Chalmers, M., LaMarca, A., and Nixon, P. Delivering real-world ubiquitous location systems. Commun. ACM 48, 3 (2005), 36–41.

This paper emphasize on the practical aspects of getting location-enhanced application deployed on existing devices without installing special infrastructure. It provides an overview of different types of ubiquitous location system. Based on two case studies, the authors reveal some interesting issues in the deployment of location-aware systems such as:

Edinburgh is an old city with many narrow streets and high buildings; its latitude of 55° north—almost as far north as Alaska—accentuates the urban canyon effects that hamper GPS.
[…]
On average, one or more access points were detected 48% of the time, and Place Lab could provide an accurate location. Two or more access points were detected for only 22% of the time. Indeed, the overall detection rate increased from 48% to 69% when excluding period of time visitors appeared to be indoors.
[…]
The game designers were surprised, for example, that rain, snow, and leaves on trees strongly affect WiFi and GPS.
[…]
The transfer of packets to and from access points can show significant asymmetry, and high packet loss can occur despite apparent network access.

Even if not standing in opposition to research aimed at improving accuracy and broadening availability, the authors suggest that we should offer pragmatic solutions while we continue to improve, adapt, evaluate the underlying technology of ubiquitous location systems.

Relation to my thesis: A reference I can use in a position paper for the workshop on Common Models and Patterns for Pervasive Computing to highlight the issues of deploying a WiFi-based location system such as CatchBob!. Besides the issues and challenges mentioned in this paper, I will add (among others things) the the uniqueness capabilities of pervasive devices.

Hazas, M., Scott, J., and Krumm, J. Location-aware computing comes of age. IEEE Computer 37, 2 (2004), 95–97.

This introductory paper to Location-Aware Computing talks about the numerous location-sensing systems that differ with respect to accuracy, coverage, frequency of location update, and cost of installation and maintenance. The authors devid them into coarse-grained and fine-grained systems.

Many similarities with A Survey and Taxonomy of Location Systems for Ubiquitous Computing.

Relation to my thesis: Which coarse-grained and fine-grained location-sensing systems ought to be used to match the location information granularity expected by a user on a specific activity. I am interested in granularity not only based on accuracy, but also in terms of timeliness.

Bell, G. and Dourish, P. In press. Yesterday’s Tomorrows: Notes on Ubiquitous Computing’s Dominant Vision. Personal and Ubiquitous Computing.

In this article, the authors advocate for developing a “ubiquitous of the present” which takes the messiness of everyday life as a central theme. The argumentation is organized around three framing points: ubicomp is already here, but does not have the form we envisioned, the futurist vision of ubicomp allows researchers for responsibilities for the present, and ubicomp is inherently messy.

Ubicomp is present
Ubicomp is essentially defined by its vision of a technological future. The literature carries the idea the ubicomp research is exploring prototypes of tomorrow’s everyday technology and everyday experience is a pervasive one. However, when we look outside of the research laboratory (based on 2 case studies in Korea and Singapore in this paper), the author look at ubiquitous computing. They argue that ubicomp is already here; it simply has not taken the form that we originally envisaged and continue to conjure in our visions of tomorrow.

The vision as excuse
The framing of ubicomp as something yet to be achieved allows researchers and technologies to absolve themselves for responsibilities for the present; the problems of ubiquitous computing are framed as implementation issues that are, essentially, someone else’s problem, to be cleaned up afterwards as part of the broad march of technology.

Ubicomp is messy, seamlessnes is a misleading vision

The seamlessly interconnected world of future scenarios is at best a misleading vision and at worst a downright dangerous one. […] Dealing with the messiness of everyday life should be a central element of ubicomp’s research agenda. In practice, though, we see that infrastructures are continually visible and must be consciously attended to in the course of everyday encounters with ubiquitous computing, from the vagaries of network access to the structure of service billing. […] Infrastructures remain messy after decades or centuries, as the user of any transit system from urban subways to international airlines can attest. […] The crux of her approach is to look at infrastructure as a relational concept; an infrastructure is an infrastructure only from the perspective of specific peoples and technologies. […] In other words, infrastructures are messy. The messiness that we experience in laboratory ubiquitous computing infrastructures is not a property of prototype technologies, of the bleeding edge, or of pragmatic compromise; messiness is a property of infrastructure itself. Infrastructures are inherently messy; uneven in their operation and their availability. […] Mobile telephony, after all, offers widespread coverage, but is neither truly ubiquitous nor truly seamless; incompatible standards, spotty regional coverage, etc., seem like obstacles that we must still overcome before the ubiquitous computing vision can be realized. But postulating a seamless infrastructure is a strategy whereby the messy present can be ignored, although infrastructure is always unevenly distributed, always messy. An indefinitely postponed ubicomp future is one that need never take account of this complexity. […] Infrastructures, then, be they networks of car mechanics, medical categories, or power sources, are never seamless in the ways in which they are put to work. They are sites of negotiation and contest, compromise and coordination, approximation and partial agreement. They are unevenly distributed and unevenly available. They are continually in flux, and brought into local stability only through active engagement and coordination. Infrastructure itself is a relational property; it describes a relationship between technology, people, and practice. […] It is not merely a dream of a world not yet realized; it is a dream of a world that could never be realized.

Relation to my thesis: The observation of ubicomp as inherently messy is at the core and a trigger of my research. Ubicomp literature mainly contain studies and prototypes embracing the seamless utopia. I wrote my paper Getting real with ubiquitous computing: the impact of discrepancies on collaboration partially in reaction to the “perfect-world” expectations around ubicomp. Weiser’s humanist vision did not fit at all my observations during the CatchBob! experiments. Bell and Dourish they take a very similar perspective as Adam Greenfield in Everyware claiming that ubicomp is here. This is probably something that was missing in my paper. I failed to notice the arrival of ubiquitous computing is rooted (at least in part) because it has been so much rooted around the idea of seamless interoperation and homogeneity.

“A Survey and Taxonomy of Location Systems for Ubiquitous Computing“, Jeffrey Hightower and Gaetano Borriello, Extended paper from Computer, 34(8), August 2001, pp.57-66.

The authors present the basic techniques used for location sensing, taxonomized location system properties (physical position vs symbolic location, absolute vs relative, location computation, accuracy, precision, scale, recognition, cost, and limitations), and surveyed research and commercial location systems that define the field. They highlight location-sensing-system accuracy as a challenge and the necessity to integrate an error factor:

We therefore suggest that future quantitative evaluations of location-sensir systems include the error distribution, summarizing the system’s accuracy and precision and any relevant dependences such as the density of infrastructural elements. (…) We strongly encourage the location-sensir research and development community to investigate how best obtain and represent such error distribution.

Accuracy and precision are defined as follow in location positioning:

The distances denote the accuracy, or grain size of the positioning information. The percentage denote the precision or how often we can expect to get the accuracy. For example reaching 1-to-3 meter accuracies 99 percent of the time.

Relation to my thesis: The limitations are defined as location system properties.

References:
Markus Bylund and Fredrik Espinoza. Using quake III arena to simulate sensors and actuators when evaluating and testing mobile services. In CHI 2001 Extended Abstracts, pages 241–242. ACM, March-April 2001. Short Talk.

The Laboratory for Ubiquitous Computing and Interaction at UC Irvine released the first version of a UbiComp reading list. This reading list is necessary for graduate students advancement to candidacy.

Relation to my thesis: keeping track on my literature review. I spotted a Hightower-Borriello article I was not aware of:

Hightower, J. Brumitt, B. and Borriello, G. 2002. The Location Stack: A Layered Model for Location in Ubiquitous Computing. Proc. of the Fourth IEEE Workshop on Mobile Computing Systems and Applications, IEEE Computer Society.

Based on five design principles extracted from a survey of location systems, they present the Location Stack, a layered software engineering model for location in ubiquitous computing.

In the future work section, they mention the challenge of uncertainty representation:

While it is clear that representing the precise nature of a sensor’s measurement uncertainty is critical, a general mechanism for this remains elusive. Traditional Gaussian representations [18] suffer from problems with nonlinear transformation between coordinate frames and the scalability of particle filters to large domains remains a challenge, although scalable state estimation techniques used in mobile robotics [8] are an excellent place to start and are the approach taken by our reference implementation.

Nicolas pointed to me the now (partially) online PhD dissertation of Jane McGonigal entitled This Might Be a Game: Ubiquitous Play and Performance at the Turn of the 21st Century. In chapter 3 Colonizing Play: Citations Everywhere, or, The Ubicomp Games, she explores the role of experimental game development in producing research insights in ubicomp (in our case mutual location-awareness in physical space, technological boundaries and design strategies to be applied) and persuading that the vision of ubicomp is worth pursuing (we do that by deploying an engaging context). She discusses our work on CatchBob! with her performance studies perspective from which the ubicomp field can learn a lot, as I did while reading that chapter. I am glad she included our paper Getting real with ubiquitous computing: the impact of discrepancies on collaboration as part of the “Are we there yet?” (in the 2003-2005 era) discourse. Converging with my impressions of the last UbiComp conference, it has become clear that this question cannot be answered because the “there” (ubicomp desired state) is very ill defined and fuzzy. From the pervasive games reviews by Jane (expect the later seamful games), I do not think they were setup to stage the imperfection, but that came up as an unexpected research outcome of the first real-world runs of CYSMN and CatchBob!

Even if not specifically mentioned, I think that it is understood that we used our game platform as an alibi for our research. I think that as suggested by Starner (2000) gameplay is perfectly suited to smoothing over the inevitable flaws and incompleteness of early technology deployment. However, it is true that, as underlined by Jane, we completely under-produced play. A trait of academic pervasive games is that neither the player nor the game take center stage, but rather the technological and interface aspects. The experiments stage an artificial (if not fake) world for the user to try out. Ciarletty (2005) describes this as the “fake it” environments and missions of so many ubicomp tests (in our case the influence of the experimental psychology approach that constraints us to defined an artificial task).

The dissertation bibliography is available as well.

Relation to my thesis: Happy that my work is cited outside my strict research community. As I wrote it very early in my PhD adventure (1 month), I do not consider (and was told) that the eMinds paper has of big scientific value, but this proves me that other communities (other research communities, industry, designers, artists) can profit to that type of outcomes. I intend my applied research to stay on this accessible track.

I could find some references I could use such as Albert Schmid (2003 or? 2005) encouraged his HCI audience to continue aggressibely pursuing Weiser’s vision, “confronting real people in real everyay environments” with more and more functional ubicomp prototypes. Schmid argues that “developing complex system isn’t a new problem. However when looking at ubicomp system, understanding the full complexity is often differnt and more difficult than in ares of more bounded scope.

References:

Ciarletta, Laurent. “Emulating the Future With/Of Pervasive Computing Development.” Online Proceedings of the Third International Conference on Pervasive Computing. Munich, Germany: 8-13 May 2005.

Schmidt, Albrecht. “Interacting with the Ubiquitous Computer.” Keynote lecture at the Fifth International Symposium on Human Computer Interaction with Mobile Devices and Services. Udine, Italy: 8-11 September 2003.

Schmidt, Albrecht, Nigel Davies, James Landay, and Soctt Hudson. “Rapid Prototyping for Ubiquitous Computing.” Pervasive Computing. 4:4 October-December 2005. 15-17

Starner, Thad, Bastian Leibe, Brad Singletary, Kent Lyons, Maribeth Gandy, and Jarrell Pair. “Towards Augmented Reality Gaming.” Proceedings of IMAGINA 2000 Conference. Monaco: 31 January-2 February 2000.