Showing posts with label ethnography. Show all posts
Showing posts with label ethnography. Show all posts

Friday, October 10, 2014

Creating a Cultural Fit: Using ethnography with users and stakeholders

ARTICLE NO. 1323 OCTOBER 9, 2014


“The power of authoritative knowledge is not that it is correct but that it counts.”–Brigitte Jordan
They stride into the arena wearing the maize and blue. They are tall, strong, fast, and confident they will conquer the world. It wasn’t easy getting here, but after countless hours of practice, weight training, and gut-busting cardio workouts, they have arrived. The price is high. It costs thousands of dollars a year to play at this level, but without access to the best coaches, facilities, and technologies, you might as well just go home.
It’s six-thirty in the morning in January, and as I watch our fourteen year old daughter’s volleyball team make their way to the court, my feelings are mixed. I’m not happy about waking up at five o’clock on a Saturday to drive for an hour through the snow. A few years ago, I’d have laughed at this elite sports scene. Now I’m a part of it. Claire is staying fit, making friends, building self-confidence, learning teamwork.
Still, it’s over the top. Our club makes me uneasy. What bothers me most is the uniforms. They are beautiful. Since our club is run by the University of Michigan, our girls are decked out in blue and gold. While a lot of teams satisfice with cotton t-shirts, we have personalized, lightweight, wicking Nike jerseys with matching shorts, warm ups, and backpacks. As our girls prepare to play, I can’t help feeling we’re on the wrong side of the tracks. And sure enough, we are crushed by the t-shirt team, just like in the movies. Later, after a day of losses, I tell Claire not to worry, it’s the first tournament of a six-month season, the team will get better.
Of course, it was all downhill from there. Our coach was a hard-ass all season. One girl was berated for not hitting hard enough. Later it turned out her finger was broken. Claire was told she couldn’t take a break even though she felt sick. Soon after she was vomiting into a bucket. The girls were taught how to trick the referee. They were instructed to lie. The coach invited them to voice complaints. Claire did so and found herself benched. The parents weren’t any better. Our alpha mom reduced other moms to tears, taunted the opposing teams, and paid for weekly private lessons with the coach. This looked like pay-to-play corruption to us, but several of the parents said that’s simply how you play the game.
The next year we switched clubs. The new one was a little less expensive and a whole lot better. When the coach told the girls it was okay to miss practice for homework, since education is more important than volleyball, he actually meant it. When we lose a game, you won’t hear a word from our alpha mom. We don’t have one. The girls practice in an old warehouse, no windows, flickering lights. It’s nothing fancy. Neither are the uniforms. And that’s the way we like it. We found our fit.

Cultural Fit

In the 1990s, as co-owner and CEO of a consulting practice, I hired and managed several dozen employees. Mostly we got it right, but once in a while we hired someone who didn’t fit. The consequence of a cultural mismatch is often compared to an immune system response. It’s not a bad analogy. The first symptom is inflammation. This pain is followed by isolation of the foreign body. But in organizations, there’s no need to destroy the antigen. Few people endure outsider status for long. They quit. At the time I thought there was something wrong with those people. My enculturation was complete. Now I know it simply wasn’t a good fit.
As a consultant for two decades, I’ve been a tourist in all sorts of cultures. I’ve worked with startups, Fortune 500 companies, nonprofits, Ivy League colleges, and Federal Government agencies in multiple countries. My clients have included folks from marketing, support, human resources, engineering, and design. Being exposed to diverse ways of knowing and doing is one of the best parts of my work. But my interest runs deeper than cultural tourism. Over the years, I’ve realized that understanding culture is central to what I do.
First, as an information architect, I must understand the culture of users. When I run a “usability test,” evaluating the system is only half my aim. I also hope to uncover the beliefs, values, andbehaviors of the people who use the system. Before imposing my own theories, I want to see how they define their world. What can we learn from their use of language and the way they sort concepts into categories? Which sources of information and authority do they trust? What is the meaning behind their behavior? For years, I’ve used lightweight forms of design ethnography as part of my user research practice. It’s helped me to better understand and design for oncologists, middle school children, university faculty, bargain hunters, and network engineers. And, as the systems we design only grow more rooted in culture, I’m convinced we must dig deeper into ethnography.
Second, as an outside consultant, I must understand the culture of the organizations for which I work. Today’s systems aren’t only integral to the lives of users, but they are progressively part of the way we do business. To improve user experience, it may be necessary to change the org chart, metrics, incentives, processes, rules, and relationships. Connections and consequences run all the way from code to culture. Software that doesn’t work “the way we work” will fail like an employee who doesn’t fit. So we must also study and design for stakeholders. In my research, I always interview a mix of executives and employees about roles, responsibilities, vision, and goals. And I’ve learned that if I don’t ask the right questions in the right way, or if I don’t listen carefully and read between the lines, I may mistake the surface for substance and invent a design that won’t fit.
Bi-cultural fit
Figure 4-1. We must design for a bi-cultural fit.
In short, the right design is one that fits the company and its customers. A mismatch on either side results in fatal error. We must use ethnography with our users and stakeholders to search for a bi-cultural fit. This is tricky since culture is mostly invisible. That’s why we should start with a map.
The right design fits the company and its customers—a mismatch on either side results in fatal errortweet this

Mapping Culture

Edgar Schein, professor emeritus at MIT and the father of the study of corporate culture, offers a useful definition.
Culture is a pattern of shared tacit assumptions that was learned by a group as it solved its problems of external adaptation and internal integration, that has worked well enough to be considered valid and to be taught to new members as the correct way to perceive, think, and feel in relation to those problems.
Culture is a powerful, often unconscious set of forces that shape both our individual and collective behavior. In an organization, culture is reflected in “the way we do things here.” It influences goals, governance, strategy, planning, hiring, metrics, management, status, and rewards.
And culture is an artifact of history. Organizational culture is rooted in the values of the entrepreneur. In the early days, as leaders struggle to build the business, the beliefs and behaviors that lead to success are internalized. Eventually they become taken for granted, invisible, and non-negotiable.
At this point, it’s difficult to decipher the culture without a compass or map. Fortunately, Edgar Schein’s model offers the orientation we need. We can use his Three Levels of Culture to ask questions about any institution.
Three levels of culture
Figure 4-2. Three levels of culture.
First, what will a visitor see, hear, and feel? Artifacts include architecture, interior design and layout, technology, process, work style, social interactions, and meetings. Who’s the boss? How can you tell? Is there music? Are people talking? What do they wear? Where do they sit? When do they eat? What makes ‘em smile? Artifacts are easy to see but hard to decode. The art on the wall is visible but what does it mean? Why’s it there? Artifacts aren’t answers, but they raise good questions.
Second, what are the mission, vision, and values? How about goals, strategy, brand? Websites, annual reports, and those colorful posters so artfully framed in the lobby offer a place to start, but interviews with insiders are the only way to the truth. Espoused values are hard to miss yet often inconsistent with behavior, which is why we need “informants” to help us see what’s really going on. If teamwork is a core value, why are individuals so competitive? If the organization is user-centered, why doesn’t anyone talk to users? It’s vital to listen carefully as insiders may not know or be willing to tell the truth. Dissonance and its justifications serve as keys to the invisible culture. Entry is earned by paying attention.
Third, what are the tacit beliefs that are taken for granted and non-negotiable? Level three is all about history. What were the ways of the founders that led to success? Are they still valid or holding us back? When we fail to seize the future, it’s often because we’re blinded to the present by the radiance of our past success. Assumptions are the bedrock of culture. They are hidden and resistant to change. As organizations grow, technologies advance, and markets evolve, friction between old assumptions and new realities is inevitable, but people don’t question what they can’t see. This is where an advisor can help. Only insiders can effect cultural change, but it often takes an outsider to sketch the map.
Intertwingled



Read more about how everything from code to culture is connected in Peter Morville's new book Intertwingled: Information Changes Everything.In it, he connects the dots between authority, Buddhism, classification, synesthesia, quantum entaglement, and volleyball.

Thursday, November 14, 2013

Becoming a Spy: Covert Naturalistic Observation


Published: November 1, 2013
“When we conduct a user experience study, we can’t help but introduce unnaturalness. … We inform participants about the study and get their consent to participate, but when people know we’re observing them, their behavior changes.”
As I mentioned in a previous column, user research is unnatural. When we conduct a user experience study, we can’t help but introduce unnaturalness. Following ethical research guidelines, we inform participants about the study and get their consent to participate, but when people know we’re observing them, their behavior changes. This is known as theHawthorne Effect. [1]
Most researchers are aware of these limitations of research, but after a while, we tend to take them for granted and may forget how strange it can feel for a participant to be in a usability lab. We may try to overcome this artificiality by going into the field and observing people in their naturalenvironments, which certainly is more natural than conducting research in a lab. But knowing that we’re observing them can still be an uncomfortable experience for participants, and we may forget that our presence affects their behavior.

Covert Naturalistic Observation

“Being covert means observing behaviors in their natural contexts without any intervention or influence by the researcher and without participants knowing that they’re being observed.”
So, how can we observe natural behavior if our mere presence affects what people do? Don’t tell them that you’re observing them. At this point, you may be thinking: Wait a minute. Isn’t that unethical? I’ll get to that, so please read on.
This type of study is known in psychology, anthropology, and other social sciences as covert naturalistic observation. It is the opposite of the techniques we typically use, which are forms of overt naturalistic observation. Being covert means observing behaviors in their natural contexts without any intervention or influence by the researcher and without participants knowing that they’re being observed.
Covert naturalistic observation isn’t a completely new idea. In addition to its use in the social sciences, it’s practiced by law enforcement agencies, spies, and amateur people-watchers all over the world. But for some reason, its use in user experience research is somewhat rare.

What Are Its Advantages?

“We can see what people reallydo rather than what they saythey do or what they show us when they know we’re studying them.”
By ensuring that participants are unaware that we’re observing them, we eliminate any effect that we have on participant behavior—the Hawthorne Effect. We can see what people really do rather than what they say they do or what they show us when they know we’re studying them. What better way to see natural behavior than to go out and observe what is really happening outside a study?

What Are Its Disadvantages?

“Being covert requires you to hide the fact that you’re observing, which makes it more difficult to observe and take notes. So it’s not very useful for studying interactions up close….”
Being covert requires you to hide the fact that you’re observing, which makes it more difficult to observe and take notes. So it’s not very useful for studying interactions up close—such as making detailed observations of a person using a user interface. It’s more suited for observing actions and behaviors that you can see from a distance.
Being covert eliminates your ability to ask questions or discuss what you’ve observed with participants. So it can leave you with additional questions and require you to make assumptions.
In the social sciences, some have criticized covert research as being ethically unsound, deceptive, and an invasion of privacy. [2] Without the knowledge that they are being observed, participants can’t provide their informed consent, and they have no choice about whether to participate. However, despite these objections, covert naturalistic observation has its advocates who argue that, in the right situations, there are ways to do this research ethically. I’ll discuss those later in this column.

When Should You Use Covert Naturalistic Observation?

“The best application of covert naturalistic observation is for studying the behavior and interactions of groups of people in public places. So it’s ideally suited for research projects for service design, workflow redesign, and process redesign.”
The best application of covert naturalistic observation is for studying the behavior and interactions of groups of people in public places. So it’s ideally suited for research projects for service design, workflow redesign, and process redesign. Some examples of questions to which covert research could provide answers include the following:
  • How can we speed up the process and improve the passenger experience of airport security lines?
  • How can we make the emergency-room waiting area a better experience for patients and those who accompany them?
  • How can we improve customers’ shopping experience and increase sales in our clothing store?
As I mentioned earlier, being covert is difficult when you need to see details close up—such as how people use a user interface. Overt methods such as contextual inquiry are better suited for those types of studies.

Tips on Covert Naturalistic Observation

Like most user research methods, covert naturalistic observation is more complex than it may seem at first glance. Just as overt observation is more than just watching people do things and asking questions about what they do, covert observation is more than just people-watching. The following tips will help you to get the most out of this method of research.

Gather Intelligence

Before you go out to observe, learn as much as you can about the domain that you’re studying, the characteristics of user groups, and the behaviors and tasks you’ll be observing. Review any previous research to which you have access. By being informed, you’ll know more about what to look for, and you’ll be better able to make sense of what you see.

Plan What You Want to Observe

“Plan what you want to learn, the questions you want to answer, and how you’re going to gather the information you need.”
If, for example, you wanted to study what people do at the airport while waiting for a flight, it would be a mistake to think, I’ll just go to the airport and see what people do. Without planning in advance what your study will focus on, you would quickly become overwhelmed by the amount of information you’d take in. This makes it especially important for you to plan what you want to learn, the questions you want to answer, and how you’re going to gather the information you need. Knowing what to look for makes it easier to focus your observation on what’s important and discard the rest.

Do Some Reconnaissance of the Location

As part of your planning, scout out the location ahead of time. Get answers to the following questions to determine where and when to observe people and how to appear natural and remain unnoticed.
  • What’s the layout like?
  • Where are the best places from which to observe?
  • What’s the activity level like at certain times of the day?
  • When is the best time to observe the behavior you want to see?
  • What are people doing and what are they wearing?
If appropriate, notify the management or security of your research. For example, if you’re observing people in an airport, the customers might not notice you, but security might. It’s better to let them know what you’re doing in advance.

Observe as a Group

There are advantages to having two or more people observe as a group:
  • Each person can observe behavior in a different location and focus on different people.
  • Each person can focus on observing different behaviors or answering different research questions.
  • You’re less likely to miss something.
  • You have someone to discuss your observations with.
If you work in a group, be sure to coordinate where each person will be, what each person will do, and what each person should focus on.

Observe During Several Shorter Sessions

Instead of one long observation session, it’s usually better to break your research into several shorter observation sessions. Doing shorter sessions has the following advantages:
  • You can see what happens over several sessions—perhaps at different times of the day, observing different people, and observing different situations.
  • You can see whether there are repetitive behaviors, situations, and patterns that recur over several sessions.
  • It’s easier to remain mentally sharp, observant, and avoid fatigue.
  • You’re less likely to be overwhelmed with too much information, so it’s easier to remember what you observed.
  • You’d appear more natural and would be less likely to get caught if you didn’t hang around too long.

Blend In

“Use a smartphone or tablet ascover—to appear like you’re doing something, not paying attention to your surroundings.”
To remain covert, you must blend in and can’t appear to be observing people. That’s easier in some situations than others, but in most situations, the following tips will help you to blend in:
  • Dress the same way everyone else does.
  • Do what others are doing.
  • Keep your level of eye contact normal.
  • Maintain a relaxed posture.
  • Use a smartphone or tablet as cover—to appear like you’re doing something, not paying attention to your surroundings. You can pretend to talk on the phone or pretend to be texting, while you’re actually observing and taking notes.

Observe Effectively

Observation sounds deceptively simple. You just watch people, right? Actually, it takes practice to observe effectively in an unstructured situation. Over time, you’ll improve your observation skills and will be able to focus on what’s important. Some things to look out for and note include the following:
  • repeated behaviors and patterns
  • unique behaviors
  • body language
  • the way people interact with other people, objects, and the environment
  • the order of people’s actions
  • similarities and differences between people

Don’t Be Obvious About Taking Notes

“Remaining covert will prevent your taking notes as easily, obviously, or extensively as you could in an overt observation such as a contextual inquiry. So don’t even try to take detailed notes. Instead, rely more on your memory.”
Remaining covert will prevent your taking notes as easily, obviously, or extensively as you could in an overt observation such as a contextual inquiry. So don’t even try to take detailed notes. Instead, rely more on your memory. Take very brief notes to capture major themes or observations that will trigger your memory later on. When the observation is over, you can type up more extensive notes from your memory, expanding on your brief notes.
As I mentioned earlier, smartphones and tablets are great cover for taking notes without your looking like you’re taking notes. To anyone who notices you, it looks like you could be using an app, texting, or writing an email message. If you must take notes on paper, make it look like you’re reading something and writing notes about what you’re reading.

Don’t Worry About Recording

You’re primarily there to observe, not record the situation. So focus primarily on observing, then if necessary, capture some photos or video at the end of the session. The purpose of these photos or video is not to record the entire observation. Their purpose is to capture important things, help you to remember what you observed later on, and show those who didn’t accompany you on the observations what you’ve observed.
To remain covert, you’ll need to take photos and record video very surreptitiously. That’s another reason to wait until the end of your sessions to take photos. If you get caught, you’ve already finished observing, so it doesn’t really matter.
Smartphone cameras are ideal for this sort of research because they’re not as obvious as traditional still or video cameras. If you turn off the flash and shutter sound, people often won’t realize you’re taking pictures. Don’trecord audio. It’s not necessary, and recording audio without consent is illegal in most places.

Do a Debrief Afterward

“Immediately after each observation session, sit down and type up your detailed notes, while the information is still fresh in your mind. First, capture whatever’s at the top of your mind….”
Immediately after each observation session, sit down and type up your detailed notes, while the information is still fresh in your mind. First, capture whatever’s at the top of your mind—such as your overall observations, patterns, themes, conclusions, and any further questions that you may have. Then review the brief notes that you took during the session to see whether those spark any additional memories of things that you should note. You may also want to draw a map or a layout of the physical space.
Once each individual observer types up his or her notes, get together with the other observers to compare your notes and discuss what each of you observed. Look for commonalities and differences. Have one person take notes on this discussion. At the end of all of your observation sessions, combine the notes from each observer, and do your analysis.

Do Covert Research Ethically

Covert research can be ethical if you’re careful not to violate any laws or ethical boundaries. Conduct covert observation only in public places, where passersby might reasonably observe the same situation. Consider whether there’s an expectation of privacy in the situation or location. For example, it may be ethical to observe customers shopping in a clothing store, but it’s certainly not okay to observe them in a dressing room.
Focus on people and situations in general, not specific individuals. Unlike a spy or police officers on a stakeout, you’re not targeting, following, or stalking a specific person. You’re situated in a public place observing what people in general do.
Be careful to protect people’s privacy by not revealing their identities in photos or videos or their names in reports. Block out their faces unless you’ve received consent to show them.
Don’t deceive participants. You’ll try to blend in, so people won’t notice you, but you’re not going under cover. If you get caught, you should just be honest about what you’re doing. Many people won’t mind and might even find it interesting.

Combine Covert Research with Other Methods

“Covert naturalistic observation … can bring up a lot of questions and require you to make assumptions. To answer those questions and test your assumptions, it’s useful to combine covert research with more traditional overt methods….”
Because covert naturalistic observation doesn’t allow you to ask questions or discuss what you observe with participants, it can bring up a lot of questions and require you to make assumptions. To answer those questions and test your assumptions, it’s useful to combine covert research with more traditional overt methods such as user interviews or contextual inquiries. You can also use traditional, overt methods first, so you’ll have a better understanding of the subject matter when you do your covert observation.
Comparing your covert and overt findings can be an interesting exercise. You can determine whether there were any interesting differences between what people did when they knew you were observing them versus when they didn’t know you were observing them.

Should You Do Covert or Overt Research?

Covert or overt observation, which is better? It depends on the situation. Both methods have advantages and disadvantages. Overt observation affects participants’ behavior. Being covert eliminates this problem, but limits what you can observe and prevents your asking questions. But as long as you’re aware of these limitations and try to minimize them, you can choose overt and covert methods as appropriate to your research and what would best enable you to understand the participants’ experience.
- See more at: http://www.uxmatters.com/mt/archives/2013/11/becoming-a-spy-covert-naturalistic-observation.php#sthash.V8LML9pq.dpuf