My PhD research plan has been validated. The current title of my thesis is: “Designing location-aware systems that manage discrepancies between the sensed physical world and its virtual representation“, but it surely will evolve. I summarize my research as follow:

Advances in mobile technologies allowed the emergence of a quantity of applications taking advantage of location. The uses of Location-Aware Systems (LAS) range from car and pedestrian navigation, finding and tracking a person, a group or an artifact, targeted marketing, local search, to the virtual annotation of the physical space (e.g. geotagging and geoblogging). However, most mobile, distributed systems and sensor technologies that deliver location information to these applications have their faults and limitations. These shortcomings create challenges for the designers and users of LAS to successfully use the contextual information promised by such technologies.

This proposal emerges from early work that shows that the uncertainties inherent to location-sensing technologies affect the usability of ubiquitous systems. It stresses on the importance of integrating the limitations of positioning technologies in a real-world use of a large-scale location-aware system.

We aim at answering the challenges enhancing the usability of a location-aware system by handling uncertainty inherent to ubiquitous technologies. Consequently we will investigate how certain a location-aware system should be in terms of location quality, location timeliness and evaluate design strategies to manage sptail uncertainty. We formulate our research question as “How to build a location-aware system that enhances its usability by handling uncertainty inherent to ubiquitous computing technologies?”. To answer it, we will rely on a classical design-science research method with an innovation building approach. We expect the outcomes of this thesis to be in the form of methods (i.e. a set of guidelines to use to manage uncertainty in LAS in order to enhance their usability) and instantiations (i.e. the realization of a real-world LAS that uses guidelines to manage spatial uncertainty).

For that purpose, we plan to use an iterative process to design location-aware mobile platform. We will hence use participatory design and fast prototyping techniques in each iterative phase of the project. The platform will provide a context for series of field studies in which we will evaluate design strategies to integrate the discrepancies of the sensed physical world and its virtual representation

Relation to my thesis: end of first year

Parush, A. (2006): Bridge the gap: Toward a common ground: practice and research in HCI, interactions Volume 13, Number 6 (2006), Pages 61-62.

Via Nicolas. This article underlines the gap between research and practice in HCI. Primarily because practitioners express difficulties in benefiting from research. HCI is a discipline concerned on the one hand with practice (design, evaluation, and/or implementation of interactive computing systems), and on the other, with the research into phenomena associated with this practice. The ability to utilize and benefit from any of the research types depends on how a practitioner defines his practical problem as a research question. The author distinguished 4 tiers in HCI research: usability, comparison, guidelines and theory.

These dimensions differs according to the “level of focus” (”range from addressing questions focusing on a specific product, to comparing between products, to searching and examining guidelines for a family of products, through to general questions on behavioral, social, organizational, and other phenomena“) and “extent of generalization”.

The ability to utilize and benefit from any of the research types depends on how a practitioner defines his practical problem as a research question. The abstraction of the question on different levels can lead one to search and find potentially beneficial research that can be applied in the practical arena.

Relation to my thesis: Currently writing my research plan I wonder how may research could have practical benefits. The 4 tiers are a good help to frame my goals and methods. Usability testing (and participatory design) to get information on the usability of my design, comparative information from literature review, guidelines from field studies (are they real behavioral research methods?), and then try to derive theoretical implications (model for a specific phenomenom?). Things are still fuzzy…

Jacques Pitrat, Comment faire de la recherche en intelligence artificielle, LAFORIA 97/06. Mars 1997

Ce papier donne quelques conseils au chercheur qui commence une thèse en intelligence artificielle basées sur le développement d’un système utilisant l’informatique. Il dégage certains points communs avec ma recherche appliquée en HCI/UbiComp.

Tout d’abord une étude d’IA devrait comporter la réalisation de deux systèmes. Le premier pour se familiariser avec le domaine et ses difficultés, sans attendre des résultats extraordinaires. Après une péridoe de plusieurs mois de décantation (écriture de papier décrivant ce qui a été fait), vient la mise en place du deuxième système, tenant compte des enseignements de la première expérimentation, qui apportera des idées vraiment nouvelles. C’est ce que je tente d’effectuer avec d’abord CatchBob! puis système possiblement un système dans le zone 22@ à Barcelone.

Deux chercheurs peuvent faire davantage de travail qu’un seul, et cela permet de s’attaquer à des problèmes qui demandent la résolution de plusieurs difficultés, chaque chercheur prenant en charge l’une d’entre elle. Je le fais déjà avec Nicolas. Thème abordé à mon dernier meeting.

Il faut savoir aller à l’opposé avec ce qu’enseignent les informaticiens. C’est-à-dire commencer à réaliser un système alors que l’on ne sait pas bien ce que l’on va y mettre (dans mon cas, faire du participatory design). Il se trouve qu’en IA la situation est en général tellement complexe qu’il est impossible de faire une analyse préalable.

Mettre son inconscient dans de bonnes conditions pour travailler. C’est-à-dire lire ce qui a été fait dans le domaine et dans d’autres domaines (trouver des analogies). Ce que je fais en aller gratter dans la géographie, psychology (spatil cognition and navigation), information retrieval, systémique et robotics. Il n’est pas bon de travailler de façon continue. Les repos permettent de digérer et d’assimiler ce qui vient d’être fait. Mélanger des travaux moins prenants aux périodes de réfléxion (littérature, rédaction).

La chronologie de la thèse peut être définie comme suit: Toute une année peut être nécessaire pour définir un sujet satisfaisant et voir comment bâtir un système capable de résoudre les problèmes qu’ils veulent lui poser. Puis la phase de réalisation et expérimentation du système est d’au moins deux ans. Il faut compter trois ans pour la réalisation d’une thèse dès lors qu’elle comporte l’expérimentation d’un système, en supposant que les trois quart du temps y sont consacré. Un pré-soutenance au bout des premières années de thèse est une bonne idée (c’est qui sera le cas pour moi avec l’obtention d’un DEA après 2 ans de PhD). C’est l’occasion de rédiger une description de l’ensemble de ce qui a été fait et de faire apparaître des insuffisancces.

Relation to my thesis: learning while doing

Today, I had a meeting with Paul Verschure. Paul is an ICREA Professor at the UPF with a background in cognitive neuroscience and neuroinformatics.

For my interest on the impact of special uncertainty, he advised me to have a closer look at the literature in spatial cognition and more precisely in spatial navigation. Somehow, his suggested to step backward to the first results and literature review I did so far. First I should gain basic understanding of spatial cognition under uncertainty and then suggest implications for design. We talked about the ways the studies are ran in experimental psychology and specifically how to carefully control conditions (manipulating uncertainty) in uncontrolled environments. it is very much possible when the control group and the experimental group are very carefully selected and managed. One challenge is not to fall into the problem of confirmation bias (interpret information according to preconceptions). In my context, I could work on three parameters (e.g. map resolution, abstraction of the data on the map, …) and then measure the performance and/or the learning.

I plan to setup a second experiment in a larger scale uncontrolled environment (because “scale” is one of the 2 most important challenge of ubicomp, next to “context”). Paul was pretty enthusiast about it and saw a lot of potential. It could actually become a real-world framework for multiple experiments (potential at the 22@ area). A scholarship is potentially available for a psychologist to work with me. Moreover, structures such as the Urban Ecology Agency of Barcelona are in deep need of data about people’s mobility and might be interested in getting involved. Such connection would match with my early digging into ABM and transportation research. I also came up some fun basic study that could involve Barcelona taxi drivers using GPS systems . To that, Paul mentioned a study in London that showed that Taxi drivers’ brains ‘grow’ on the job due to the navigating they do (”The posterior hippocampus was also more developed in taxi drivers who had been in the career for 40 years than in those who had been driving for a shorter period.”).

On the side, Paul introduced me to Bayesian Inference and how users have multiple sources of information (in the context of spatial uncertainty).

Relation to my thesis: facing experimental psychology and first thoughts on the resources available for a second experiment. I will dig in the spatial cognition literature.

I have 3 papers due for my doctoral school courses on Information Retrieval (Geospatialweb and user context), Artificial Intelligence (not sure yet, but something related to Reinforcement Learning, and Research Seminar (Context driven information supply). Jennifer Widom’s Tips for Writing Technical Papers offers relevant information to structure my assignments.

Relation to my thesis: Learning to write scientific papers

Computing science is an immature discipline and HCI still looking for its “theory”. Chris Johnson wrote 2 papers on What is Research in Computing Science? and What is a PhD in HCI? providing a high level introduction for PhD students to grasp the problems (applying standard to scientific empiricism to computing science) and challenges of the 2 disciplines.

I was aware of implementation driven research, mathematical proof techniques, empiricism, and observational studies as research methods in the field of Computer Science. However I discovered Hermeneutics as an alternative research methods that addresses the formality gap (i.e. the distance between mathematical models and reality). It stresses upon the analysis of a final implementation closely resembles proof by demonstration (field trials with real sets of data on existing architectures).a

One main problem of doing research in HCI is that its inter-disciplinary nature makes it difficult to identify clear guidelines or standards. Neither is it clear to know if HCI is a craft or an engineering discipline. Nevertheless, Johnson acknowledges a set of criteria that can be used to assess the quality of PhDs in HCI:

• A Grounding in Experimental Techniques: Candidate shows a proper grounding in experimental techniques lab-based versus contextual techniques
• A Grounding Design: the candidate contributes to the design and implementation of interactive systems.
• Understanding of inter-disciplinary research: demonstrate an understanding of inter-disciplinary research.

Of course, a PhD students should prepare for the standard questions that are asked at almost all HCI vivas:

Experimental work must be defended against accusations that it fails to explain real-world behaviours. Design innovation must be defended against criticisms that few industrial designers use the products of academic Computer Science. Inter-disciplinary research must be defended against technical criticisms drawn from each of the parent subjects.

Relation to my thesis: I am still evaluating the validity of the different approaches in computer science and understand how they apply to HCI. Empirical proof techniques is the obvious pick (with a dose of implementation driven research and observational studies) the goal being to map experimental theses with real-world analysis of situated interaction. Maybe my engineering background (and approach) will facilitates this bridge between designers and exploitable theoretical results (a current shortcoming in HCI).

The research bible On research methods, by Järvinen Pertti; Opinpaja Oy 2004, is very resourceful to me in understanding the research process, selecting a research approach and applying design-science research.

There are four possible purpose of science, namely to describe, the explain, to predict and to control. However, we shall not restrict to them, because they do not cover the study off whether we can or cannot build a new artifact. The purpose of such a study can be understanding the system, its re-engineering and evaluation.

In How to select an appropriate research method in ergonomic studies?, Jarvinen provides a taxonomy of research methods.

My research ideas tends to study their utility as innovation by means of building and evaluation. In building a new innovation, utility aspects are striven and a particular development method is applied. In evaluation of the innovation, the realized final state is compared with the desired goal state, and maybe some criteria are used and some measurements performed. This process is called design science. The goal of design-science research is utility (while behavioral-science research aims at truth).

Research-science research is about answering the questions: Can we build a certain innovation and how useful is a particular innovation? The research questions contains verbs like: build, change, improve, enhance, maintain, extend, correct, adjust, introduce.

“The mission of design science is to develop knowledge for the design and realization of, i.e. to solve construction problem, or to be used in the improvement of the performance of existing entities, i.e. to solve improvement problems”, in other words, to implement some innovation.

The mission of design-science can also be seen as to develop design knowledge.

Design knowledge concerns “three designs: an object-design, the design of the intervention or of the artifact; a realization-design, i.e. the plan of the implementation of the intervention or for the actual building of the artifact; and a process-design, i.e. the professional’s own plan for the problem solving cycle, or, put differently, the method to be used to design the solution to the problem.

The design sciences are not too much interested in what is, but more in what can be.

The integral outcome of both construction and improvement is called a technological rule. Construction has four outcomes (construct, models, methods and instantiations) and two research approaches (build and evaluate).

A technological rule is defined as a chunk of general knowledge, linking an intervention or artifact with a desired outcome or performance in a certain field of application.

A major breakthrough rule occurred with the testing of the technological rule and then grounding on scientific knowledge.

Guidelines as requirements for effective design-science research are:

Design-Science Research guidelines from Alan R. Hevner, Salvatore T. March, Jinsoo Park, Sudha Ram: Design Science in Information Systems Research. MIS Quarterly 28(1): (2004)

The building process
Building is a process of constructing an artifact/innovation for a specific purpose. Early in the process extracting multiple case study is a good research practice to uncover technological rules already used in practice. In the developing multiple case study, the technological rules are developed and tested by researchers in close collaboration with the people in the field and often in the context of application. Following a reflective cycle,after each case, the researcher develops knowledge that can be transferred to similar context on the basis of reflection and cross-case analysis.

Design science products are of four types:
Constructs: or concepts from the vocabulary or language of a domain
Model: a set of propositions or statements expression relationships among constructs
Method: a set of steps (an algorithm or guidelines) used to perform a task
Instantiation: the realization of an artifact in its environment

The purpose of of the construction process is to achieve a movement from the initial state to the goal state (part of the specification process). The descriptive model of the initial state may need to capture the structure of reality in order to be a useful representation. The normative model of the goal state represents how things how to be.

A practitioner ought to describe the building process in detail, argue the selections and explain the decision. The originality of the solution and its superiority to the known solution must also be demonstrated.

The evaluation of the construction results
The research contribution lies on the novelty of the artifact and in the persuasiveness of the claims that is it effective. Evaluation is a process of determining how well the built artifact performs. It the research outcome is totally new, the actual performance evaluation is not required at this stage. When the old outcome exists, significant differences between the old construct, model, method or instantiation and the new one.

Metrics are needed in the evaluate activity to define what the research is trying to accomplish. Proposed universal metrics target all types of artifacs (i.e. constructs, models, methods and instantiation):

• Constructs: evaluation ten to involve completeness, simplicity, elegance understandability, and ease of use
• Models: evaluated in terms of their fidelity with real world phenomena, completeness, level of detail, robustness, and internal consistence. Model have to ignore things exactly because they view the world at a level of abstraction. The models can be represented in many way, physically, mathematically, pictorially etc
• Methods: operationally (the ability to perform the intended task or the ability of humans to effectively use the method if it is algorithmic), efficiency, generality and ease of use.
• Instantiations: efficiency and effectiveness of the artifact and its impacts on the environment and its users. This includes unplanned changes with positive and negative unanticipated outcomes that accompanied these changes.

Once metrics are developed, empirical work may be necessary to perform the evaluation. Constructs, models, methods and instantiations must be exercised within their environments. Often this means obtaining a subject group to do the exercising. Often multiple constructs, models, methods or instantiations are studies and compared.

Action Research
Action Research assumes that both building and evaluating sub-processes closely belong to the same process. It is the production of knowledge to guide practice, with the modification of a given reality occurring as part of the research process itself.

Relation to my thesis: I better understand how I should formulate my research questions and the process to answer them. Design-science research, in its building phase, has many similarities with engineering that I am familiar with (understanding of the initial state, requirements settings, definition of the goal state, instantiation, technological selection, iterative process, fast prototyping, … ). I am less aware of a scientific evaluation process.

As part of my investigation that explores the people perception of discrepancies in the context of collaboration supported by ubiquitous environments, my object-design should aims at improving the individual and group performance in and real-world uncertain ubicomp systems. First I will need to understand uncertainty from the literature and my own experiments (set the initial state), then build case studies (i.e. set the goal state to reach, different approaches and contexts), test field and evaluate them.