Milestone 3

Page

Introduction

Recap

By the end of the second milestone, we had completed a formative study which involved deploying a cultural probe to investigate how people typically make sense of new surroundings, including new things, people, and behaviors, especially in unfamiliar cultures and environments. Based on the results of the formative study, we had generated three refined concepts:  pervasive “information displays” installed in various environments to help people learn about new things, an “information mapping application” via wearable smart glasses to help someone relate new objects to his/her past knowledge, as well as a mobile “scavenger hunt” to help people discover and explore a new environment.

These three refined concepts primarily centered around the learning and discovery new “objects,” while leaving the question of how technology might support the learning of new behaviors and actions largely up in the air. Upon a revisitation of the formative study findings, combined with feedback from fellow classmates about the existing concepts, it became clear that learning unfamiliar norms and behaviors is more difficult for people than learning about new objects — thus presenting a more interesting opportunity for technology to provide assistance. The exploration of new objects involves relatively low risk, and it seems that people are fairly good at doing this already. Learning and adapting to new behaviors in unfamiliar cultures, on the other hand, is a more laborious process, with higher risk of bad consequences (offending people, etc). Sometimes people are unaware that they are violating cultural norms (these points are summarized in Findings 3 and 4 of the formative study).

Concept Evolution

With the above points in mind, the team began to further refine the concepts. Based on feedback from Milestone 2, the “scavenger hunt” seemed to be the most promising with respect to demonstrate-ability, feasibility, and innovation. However, the scavenger hunt, as it currently stood, did not seem to be particularly useful. While it made traveling to new cultures into a game, and therefore more fun, it focused less directly on helping someone acclimate or learn about unfamiliar cultures. We therefore took the “scavenger hunt” idea in a different direction: instead of pushing information to the user about suggestions of activities to do, it would provide information and tips about cultural norms and behaviors in situ . This way, a user could be better prepared and informed as to acceptable interactions with others in an unfamiliar culture — something that, from our formative study, participants identified as difficult to learn. We have dubbed this new concept Warwick.

Experience Prototype

Overview

For our experience prototype, we decided to conduct several user enactments to test a low fidelity version of Warwick in a simulated, fictitious culture with fictitious norms. In these user enactments, the participants were asked to navigate the norms of an unfamiliar culture with the aid of Warwick. By creating a fictitious culture, it was guaranteed that  each participant would be unfamiliar with that culture’s behaviors. Furthermore, this allowed us to better control the norms and behaviors of the simulated environment, minimize noise which might have resulted from participants’ prior knowledge of existing cultures, and allowed us to focus on understanding the variables we adjusted with respect to the system’s design.

The fictional culture we created was the island country a Capella. In Capella, the norm when dining out is to tip one’s waitress at the beginning of a meal, as an unspoken agreement of how much the diner wishes the waitress to attend to him throughout the meal. The higher the initial tip, the better the service. When visiting a local Capellian’s home, the strictly acceptable behavior is to present your host with a gift of flowers, else the host might consider the guest ungrateful.

Research Questions

There were several dimensions of Warwick we were interested in investigating. Based upon Finding 1 of the formative study, we learned that different people have different levels of tolerance for ambiguity when it came to discovering new things. Thus, it was important to explore the level of ambiguity in the information the system provides to users. Further, it was important to understand the ideal level of proactivity that the system should encompass in order to provide users with the most satisfying experience. Finally, we were interested in learning whether or not people’s preferences about ambiguity and proactivity levels would change depending on the contextual situations they are in.

We identified two primary research questions and two secondary questions:

  • What is the ideal level of proactivity for system to intervene with advice about correct cultural behaviors?
  • Does the level of proactivity vary between social situations with lower risk (dining at a restaurant), versus social situations with higher risk (offending an important business client)?
  • What is the ideal level of ambiguity in the information/advice that the system provides the user?
  • Does the level of ambiguity vary between social situations with lower risk (dining at a restaurant), versus social situations with higher risk (offending an important business client)?

Speed Dating Matrix

The dimensions identified in our research questions were translated into a Speed Dating Matrix, as outlined by Davidoff et. al. in Rapidly Exploring Application Design Through Speed Dating. These dimensions included the proactivity of the system (high, medium, low), the level of ambiguity in the information presented by the system (high, medium, low), as well as the relative pressure of the scenario (dining in a restaurant: a public scenario with relatively low pressure, as well as visiting the home of a high-end business client: a more intimate scenario with relatively higher pressure).

We chose five scenarios to carry out in user enactments that differed enough from each other to give us a wide spread of data. We also chose some scenarios which pushed the boundaries of what might be socially acceptable in order to explore and discover the tensions.

A complete and detailed documentation of our speed dating matrix, chosen scenarios, and research questions we tried to answer with each scenario, may be found in Milestone 2.5.

Recruitment

We sent an email to umsi.all.open to recruit people who might be interested in participating. We recruited five participants, between the ages of 20-32, of varying cultural backgrounds, gender, and travel experiences. Each participant received a $5 Amazon gift card.

User ID

Age

Gender

Native Country

Occupation

Traveled to other countries?

Phone type

Level of Technology Use

P1

22

M

China

grad student: HCI

yes: USA

iPhone 4

medium

P2

28

F

USA

grad student: HCI

yes: Canada, Australia, Bahamas, St. Maarten/St. Thomas, Germany, Italy

iPhone

high

P3

29

F

USA

library employee; teacher; tutor

yes: extensively abroad, spending considerable amount of time in several Eastern European countries as well as other European nations and New Zealand

Android

medium

P4

24

F

USA

grad student; IAR, HCI, HI

yes: France, Canada, Mexico

Windows 8

High

P5

32

M

Cyprus

grad student: communication studies

yes: UK, Iceland, Lithuania, Czech Republic, Greece, Israel, USA

Android

High

User Enactments

We conducted a within subject study in which each participant took part in all five user enactments in order to better understand how one participant reacts to the variations in each scenario. User enactments took place in various study rooms and classrooms within North Quad. Each room was first converted into a restaurant, complete with table cloths, plasticware, plates, cups, menus, even Capellian “food” (paper print outs of fantastical food items). Then, each room was converted into a home, complete with a “bay window,” “plants,” and decorative art.

Detailed roles and scripts for the user enactments may be found here.

IMG_0608.jpg

A Capellian waitress takes an order from a study participant, as a lone diner eats and considers life in Capella.

IMG_0630.jpg

A participant, seeing her business deal is going awry, attempts to offer her client a prop phone as a gesture of good faith.

IMG_0669.jpg

A Capellian waitress takes an order from a lone diner, as the participant reads over the menu waiting for service.

menu-items.jpg

Props used in the study.

propaganda-posters.jpg

Capellian propaganda posters.

phone-proto.jpg

Lo-fi prototype of the system.

Methods for Analyzing Results

After each interview, the note taker created affinity notes from the interview data. These affinity notes were then coded as “Quotes”, “Insights”, and “Questions”. Quotes were the statements said by the participants. Insights were derived by reading in between the lines and recognizing patterns from the participants’ interview data. Questions comprised of both those asked by the participants and ones that the interview team had after synthesizing the affinity notes from the interview data.

Next, the entire team analyzed these synthesized and coded affinity notes and highlighted the most important ones. Following the highlighting process, semantically similar notes were categorized and bucketed together.  Next, each bucket of notes was re-read and common themes were extracted derived from them. After the complete process we had five distinct themes. These themes were then distilled further into concise findings.

Since we applied a bottom-up process to arrive at our findings, we looked back at the affinity note buckets and came up with the evidences that supported each finding. Once, we felt comfortable with our findings and the corresponding evidence, we started synthesizing the design implications from each of the findings.

Study Results: Findings & Design Implications

Finding I: All participants reacted negatively to scenarios with high ambiguity and high proactivity.

Evidence

Participants were both wary and uncomfortable when our proposed system acted on their behalf and didn’t specify what actions it performed or why it performed them. In these scenarios, the system merely informs the participant in general terms that they have made some cultural breech of etiquette, and that it has corrected their mistake for them. Participants were generally inquisitive and sought more information. They subsequently were frustrated at not being able to receive any and required two pieces of knowledge to be satisfied: 1) what the breech of etiquette was, and 2) what the system did to fix it. Additionally, participants wanted to know exactly what the norm was so they could personally have the freedom to choose what to do with that information.

One participant saw a highly proactive system that fixes their mistake as impeding on their ability to actually learn about different cultures and their norms. P3 stated: “I want to learn, why will the system not allow me to learn?”

Two participants noted they would never want the system to do any for them that required financial decisions, with reference to one scene where the system tips the waitress for them. P2 responded to this by saying,  “I would never let this device do financial transactions on my behalf.”

Design Implications

The system should not be highly proactive and highly ambiguous. It should provide adequate information when explaining cultural differences to the user. Moreover, there should be a “Read More” button to support additional probing.

If there are any exceptional cases in which our system will be highly proactive, it should, at the very least, asking participants permission before performing actions on their behalf. This would provide more comfort to the users.

Finding II: Most participants preferred being provided with information that was concise, directive and had a high level of clarity.

Evidence

Participants generally preferred scenarios with low ambiguity over ones with high ambiguity. Most were most at ease when they had exact information that was very concise and to the point. We found that this was especially true with respect to behaviors and actions. In scenarios where our participants were informed of some cultural difference in Capella, most always wanted to know exactly what it was, and often even probed for additional information. Only one participant, P3, preferred medium ambiguity over high ambiguity.

In the case of objects, participants preferred the information to be more neutral and less directive. In the case of objects, participants preferred the information to be more neutral and less directive. In one scene, the system suggests to the participant: “Do not try the gruel under any circumstances.” P5 asked specifically, “Why shouldn’t I try the gruul?” P2 found the suggestion disruptive, and said she would try the gruul no matter what in a real scenario. This result is also supported by our formative study.

Design Implications

Information presented about normative behaviors will be clear, concise and directive. Information presented about objects (food, etc) may be less directive, and more neutral and exploratory, so a user may decide for himself how he wants to proceed when encountering a new object.

The system should generally provide neutral, unbiased information to the user. If the system does provide non-neutral information, like crowd-sourced suggestions or comments from other users, it should be clearly marked as such.

Regarding P3, who preferred medium ambiguity, the system could cater the level of ambiguity depending on a user’s tolerance for it. This could possibly be done through a short questionnaire during on-boarding to help assess their preferences.

Finding III: There was an even split between preference of low and medium proactivity and the preference was largely context dependent.

Evidence

Although no one preferred high proactivity, there was an even split between the preference of medium and low proactivity. We found that this preference varied from act to act, when questioned, participants revealed that the preference is largely dependent on the situation they are in.

Design Implications

From our participants we learned that an appropriate level of proactivity for the system depends on the context of the situation, ideally different situations should be weighted with different levels of proactivity. A determining factor in this is, as one participant described, a situation’s “impact level.” The impact level of a situation has many factors, including, but not limited to:

  • Repercussions – The severity of the consequences for inaction or incorrect action with respect to both the individual and others
  • Immediacy – The level of urgency of an individual’s response
  • Level of specificity – This factor refers to how difficult it is for an unaided individual to determine information about a situation, whether it can be more readily understood or is something more intangible and unspoken that cannot be discerned from direct observation
  • Level of difference – The situation sits on a spectrum which ranges from unfamiliar to familiar – either it is far removed from an individual’s native culture, or it is more familiar, meaning it is different but still comparable

Taking into account these factors, a situation with a low impact level is one that has little to no consequences, does not necessitate an immediate response, and an appropriate response can be easily observed and extracted from context or environmental clues. In a case like this, a system with lower proactivity would better suit the users. Ideally, by using the the factors above, our system would be able to determine a situation’s impact level and adjust the proactivity accordingly.

In addition to the impact level, we also learned from one participant the desire for the level of proactivity to decrease as their exposure to a new culture increases. After assisting users through an initial acclimation period, some users would like the system to fade more into the background. This functionality would have the added benefit of allowing the users to immerse themselves in a culture without being constantly barraged by notifications. In a manner of speaking, the system would learn to take off users’ training wheels when they become more self-reliant and confident in navigating the new culture in which they find themselves.

Finding IV: With respect to timing, participants wanted to be prepared in advance for unfamiliar situation.

Evidence

In line with our formative study, we found that many participants in our scenarios would prefer to be prepared before entering the unfamiliar situation. Participants remarked that if they were actually traveling to Capella, they would have done research before arriving.

Design Implications

This finding primarily deals with timing and when users should receive information from the system. Our system should be able to determine the appropriate timing of when to provide specific information to the users, whether it should be before, during or after a certain situation or encounter. To do this, the system needs to have a clear taxonomy to decide which of these three timings is most appropriate and when. To determine this, the system could potentially use the “impact level,” as described in Finding III.

We found that most individuals would do research before going to a new place. The proposed system could take this into consideration, and attempt to augment this existing behavior. This way, the system could take the burden of research off the users and take the initiative to notify them of general information about where they’re going before they go. Although, the proactivity level of this feature needs to be considered.

Finding V: The source of information, with respect to its accuracy and credibility, was not a concern to participants.

Evidence

While being debriefed after the scenarios, participants were asked what their feelings were regarding the validity and credibility of the information the system presented to them. In general, our participants’ responses implied that this wasn’t much of a concern to them and much of the time they didn’t even consider it. One participant (P1) mentioned that because the information wasn’t “ridiculous,” or that it seemed plausible, they trusted it and it raised no critical doubt in their mind. Another (P4) even went as far as to say they would implicitly trust the source of information and even speculated that they would “take the system for granted.”

Design Implications

This finding reveals that considerations and judgements of value by users regarding the source of information and its credibility are rare to occur. Furthermore, in designing our system, the users’ trustworthiness of the information being received is not a vital consideration.

Ideation and Selection

According to the findings stated above, we developed some criteria to analyze and assess our ideas. Each of them gives a different aspect to investigate the system with and demonstrate its functionality.

Ambiguity

The criterion of ambiguity indicates how much information a user would like to receive from the system. In our experiment design we had three levels of ambiguity, high, medium and low.  These respectively correspond with a low amount of information, a medium amount of information, and detailed information. A good system would determine the level of ambiguity which users prefer and give appropriate amount of information.

Proactivity

Proactivity represents to what extent the system will act for the user. High proactivity indicates that the system will automatically perform functions for the users to solve problems while low proactivity means the system will do little for the users. There is always a trade-off between convenience and security concerns with regard to different levels of proactivity. A situation’s context is one of the key factors in determining an appropriate level of proactivity.

Intrusiveness

A user-friendly system is one that does not intrude on a user’s normal life. Users may feel offended if the system does something that they feel is uncomfortable or if it does something without their consent. Therefore, an ideal system should respect users’ opinions toward what kind of role it should play in users’ lives.

Adaptivity

All the criteria we defined above has the attribute of adaptivity. Different people have different personalities. Hence, they have distinguished opinions towards ambiguity, proactivity and intrusiveness. A smart system would adapt to a person’s characteristics and perform different levels of these factors based on that information. It may be also adaptive to the social context, e.g. using a user’s prior knowledge about the norms, etc.

While discussing the findings and insights from the experiments, our group applied these criteria to analyze our system to determine what amount of information the system should give and how the system should perform for the users. Then each individual generated storyboards with different proposed designs. We together talked over the pros and cons of each, and came up with a refined storyboard and system proposal.

System Proposal: Warwick

Warwick is a mobile phone application which helps users navigate unfamiliar cultures and norms. Warwick provides users with helpful tips and information regarding the behaviors, norms, and other aspects of unfamiliar cultures, so that users are able to better learn, adapt, and acclimate into new environments.

The information that Warwick provides users comes from a variety of sources. The predominant information source will be crowd-sourced data from other travelers, though Warwick may also aggregate data from search engines such as Google, Wikipedia, or Yelp. Warwick will tailor the information that each user sees by finding tips from others “like the user,” or voted popular by “others like the user.” (For instance, from other travelers who are also from the user’s home country.) This way, the user sees relevant data from others with whom he shares common ground, and who might be familiar with the social contexts the user is experiencing. Furthermore, information presented about normative behaviors will be clear, concise and directive. (Finding II) Information presented about objects (food, etc) may be less directive, and more neutral and exploratory, so a user may decide for himself how he wants to proceed when encountering a new object. (Finding II)

Warwick is an intelligent, adaptive, mobile system which will provide users with just the right type of information, at just the right time. First, through a combination of mobile sensing technologies and machine-learning techniques, (such as GPS, calendar info and Google search info, etc) Warwick will be intelligent enough to understand when a user is about to enter, or is currently immersed in, a possible unfamiliar cultural environment. Based on this information, Warwick will determine what type of information to push to the user, and at what level of proactivity, and at what time. Warwick operates using two levels of proactivity: low (user initiates action by opening the application and searches for information) and medium (Warwick alerts the user by sending a standard push notification). The type of information (this includes what type of content, the specificity of that content, etc) level of proactivity, and the timing of the information delivery is determined based upon a situation’s “impact level.”

A situation’s “impact level” (a term coined by one of our participants) refers to the level of repercussions, consequences, or effects on other people, if any norm(s) were to be violated. For example, tipping the incorrect amount in the United States might be considered “low impact,” as the only consequences that might result are poor service and a mildly offended waitress. Tipping a waitress in New Zealand, however, might be considered a  “high impact” situation, as tipping (according to the experiences of P3) might be taken as a sign for sexual solicitation. A situation’s “impact level” may be determined by a number of factors. Further work and research needs to be conducted in this area, but these factors could include: repercussions, immediacy, level of specificity and level of difference. (Finding III)

Therefore, a situation with a higher “impact level” would indicate to Warwick to use a medium level of proactivity, and to provide more directive information compared to a situation with a lower “impact level,” which would require less proactivity from Warwick. Moreover, the proactivity of Warwick for any one given situation will also adapt based on how familiar the user is with the new norm. Upon first entering a restaurant in an unfamiliar country, Warwick might alert the user proactively of acceptable tipping behaviors in that country. Once the user has entered his second and third restaurants, however, Warwick should decrease its level of proactivity.

Refined Storyboards

The following five scenarios demonstrate Warwick’s adaptive nature. Warwick is able to adjust its level of proactivity, timing, and type of information, according to a situation’s impact level, as well as the user’s familiarity with the situation.

Scenario 1: Booking the trip

Warwick reacts to a low impact situation with low proactivity, provides general information, and presents information well in advance of Mike’s trip to a new culture: the island country of Capella.

scen-01.jpg

Scenario 2: A week before departure

Here, Warwick reacts to a high-impact scenario with medium proactivity, provides specific information, in advance of the scenario taking place.

scen-02.jpg

Scenario 3:  Arrival in Capella — dining out

Warwick reacts to a medium impact scenario with medium proactivity, high specificity in information, during the situation.

scen-03.jpg

Scenario 4:  The day of business meeting

Warwick reacts to a high impact situation, provides specific and directive information, shortly before the situation takes place.

Scenario 5: Dinner

Warwick reacts to a low impact situation it knows the user is familiar with using low proactivity.

 

Demo Proposal

Our demo will try to bring out the entire experience of easing people into new environments where they might not be aware of certain customs and traditions. The demo would have the user engage with a mobile application in an unfamiliar setting, following a written script.

Since, in a normal situation, a traveler/ tourist would encounter different situations that he or she might not be aware of with varied forms of repercussions to the users actions, our primary goal is to simulate multiple environments that the user may encounter as a traveler. It is important that we showcase as to how our system would be intelligent enough to prepare the user for untoward circumstances and at the same time helping the user cope with small cultural changes without intruding the experience of the user much.

We would not be able to use a fully functional high fidelity prototype, as it would be too expensive to have a context aware system that pulls data from your location, calendar entries and other cloud-based services. Also, within the timeline of this project, it would not be probable for us to do so.

Hence, we would be using a mobile phone that would push notifications as and when the user would come across a new environment, sensing the right time depending on whether the situation is a high or a low impact situation. We would have a medium fidelity prototype of the app within the scripted environment. Projectors along with other props could be used to project different scenes onto the walls to give the user a perspective of being in a certain environment. A confederate will make buzzing noises from a speaker to imitate the phone buzzing. This confederate will also be able to push screen onto the users screen via screen sharing capability based on the users interactions.  The user would be able to interact with the app either before entering the new social situation or during it, based on the impact of the situation. The app will then give cues to the user to help gel into the new environment. The ambiguity of these cues would be completely dependent on whether the environment is a high impact one or a low impact one. Based on the ambiguity of the cue, the user would be able to find out more about that particular social difference by interacting with the system further.

Furthermore, the demo would help us visualize certain key aspects that were not reflected in the user enactments. For instance, the fact that there are people studying you does make the participant conscious and he/she would not react naturally. Having other people being there in the setting may cause a participant to act in a certain way.

Demo Experience Storyboard

 

Conclusion

At the end of this milestone our group is ready to demo the idea that we have with detailed

emphasis on how it will be implemented in a real time scenario. We will be leveraging the mobile phone technology to demo the use case of a context aware system that helps travelers and tourists know and adjust the norms and traditions of a new cultural setting.

By using experience prototyping methods, our group was able to test the system for varying levels of proactivity of the system and ambiguity of the information that the system presents to the user. We used the speed-dating matrix to analyze all these different scenarios and came up a system that was an amalgamation of multiple ideas across the matrix.

The mobile device carries a context/ culture aware app which fetches information from the travelers schedule, travelers location, the web and from other fellow travelers’ inputs to inform the person about the certain unknown norm at a new place he/she is going to. The testing phase helped us to understand the systems and travelers behaviors in two situations, viz. high impact and low impact. High impact are situations where a wrong decision or action on the part of the traveler can have outrageous consequences and low impact are situations where the consequences are not that grave regardless of users actions or decisions. This helped us gauge how much information as well as when it should be presented to the user in these different situations.

Moving forward, we need to be careful about the implementation of this system, especially in the social context. We did analyze most of it in the user enactments, but in a controlled and experimental setup that involves participants, who are being studied, might involve lot of biases. Also there were some other questions that arose, for instance the fact that we did not give participants enough time between each enactment might have lead them to act in a certain way.  Some skewed results might have crept into the findings, as there were points where the script did not match the screens on the prototype completely.

But overall there have been no major changes in our idea to implement the system and the major functionalities; rather the experience prototyping stage helped us validate most of our assumptions and findings from previous studies.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s