Video Diaries: A Method for Understanding New Usage Patterns

Go to article

By Paul Bryan

Published: January 23, 2012

In my new column, UX Strategy, I’ll explore the growing field of user experience strategy, which combines business strategy with user experience design to build a rationale and a road map for guiding an organization’s UX efforts. This column will address methods and practices that UX Strategists can use to collect data, formulate personas and interaction models, document UX strategies, and create UX road maps.

“UX teams are feeling the pressure from all sides to integrate and innovate as they design user interfaces that must span multiple channels such as the Web—on multiple browsers—smartphones, tablets, game consoles, and kiosks.”

 

These are exciting times. The business of user experience is evolving rapidly. In a very short period of time, the complexity of user-interface considerations has multiplied. New product and service introductions occur so fast that trying to keep up with last month’s announcements is like drinking out of a fire hose. Consumers have so many options at their fingertips that the vast majority of them have barely begun to exploit the resources that are only a tap or a download away.

UX teams are feeling the pressure from all sides to integrate and innovate as they design user interfaces that must span multiple channels such as the Web—on multiple browsers—smartphones, tablets, game consoles, and kiosks. Airport kiosks, iPods, ATMs, and video game consoles have convinced consumers that every digital experience should be simple, self-contained, and let them achieve their desired goals with just a few taps or clicks. Layer onto all of this the need for integration with social networks and the ability to factor in location, and the number of moving parts increases dramatically. It’s like going from playing chess on a two-dimensional board to playing in three dimensions.

Zooming Out

“A common solution is to introduce a uniform set of standards and patterns that work across all platforms. This is an understandable attempt to reduce complexity and keep up with the demand for UX design work.”

 

In response to all of the complexity that change has thrust upon us over the last few years, the clients and UX teams for whom I consult are often tempted to zoom out to get a bird’s eye view of the overall landscape. A common solution is to introduce a uniform set of standards and patterns that work across all platforms. This is an understandable attempt to reduce complexity and keep up with the demand for UX design work. It also fits in well with the dominant agile ethos.

But there is a problem. Satisfying this demand is not simply a question of thinking about and producing solutions for more of the same types of UX issues. We need to think about these for sure, but we also face an entirely new set of UX issues. We don’t simply need to design more user interfaces. The existence of similar user experiences on various devices and in different channels influences the patterns that designers employ when designing a particular user interface. We need to design user interfaces that take into account the existence of similar interactions and information on other devices, with the goal of meeting related, but different needs. The theory of relativity is making its way into UX strategy!

Zooming In

“As consumers become comfortable with emerging technologies, they are trying new combinations of interactions that help them to optimize the usefulness of these technologies.”

 

As consumers become comfortable with emerging technologies, they are trying new combinations of interactions that help them to optimize the usefulness of these technologies. For example, a woman shopping in a store might scan a dress tag using her iPhone’s camera, view the dress on a virtual model of herself, and post the image to her social network for advice. While waiting for a response, she could save the dress to a shopping list she’s creating on the retailer’s Web site, then check other sources for a similar dress while browsing on her iPad. Later, while watching TV, she could pull up the dress she found earlier. She might find a similar dress elsewhere at a lower price.

Then, she might even tap an Accessories link to see images of coordinated accessories superimposed on the dress her virtual model is wearing. She could then look for various sources of the products making up the whole outfit, balancing price with the reputations of retailers. This isn’t really a far-out scenario; nor is it just more of the same old ecommerce user experience. It’s both more and different.

The most common user research methods—usability testing, A/B testing, and online surveys—are not adequate for fleshing out usage scenarios in the coming multichannel world. In-depth interviews that take place within a user’s context—known as in-context interviews or contextual inquiry—are not sufficient either. They offer a deep, but narrow slice of reality, are too brief, and occur in only one setting. It takes a holistic immersion in the consumers’ world, with all of its devices and complexity of options, by people who understand design, user research, and consumer decision making. This is not research to enable the incremental change of existing user interfaces. It’s about designing an overall user experience that spans multiple, complementary, yet different user interfaces. 

One research method I’ve used extensively in the past couple of years to gain a deep, inside look at consumers’ rapidly evolving interactive behaviors is the video diary. Instead of my team of user researchers’ querying consumers about their usage patterns, we empower consumers to produce their own stories, over a period of time, as they use various resources and devices to complete real-life tasks. Using readily accessible digital tools, participants self-document their behavior in the context of activities we are studying.

The Video Diary Protocol

“The video diary protocol is different from most user research protocols, because participants lead themselves through it rather than a researcher’s guiding them.”

 

The video diary protocol is different from most user research protocols, because participants lead themselves through it rather than a researcher’s guiding them. To be successful, the protocol—or guide as it’s sometimes called—must be simple to understand, yet very thorough, giving clear instructions for every step. It’s also necessary to formulate the guide to get people to talk about issues that can help you solve the design problems that your client commissioned a research program to address.

We structure our video diary protocols similarly to a printed diary. It’s divided into days, with a large heading at the top of each page for DAY 1, DAY 2, and so on. Below each heading are the questions corresponding to that day. Although, in reality, participants usually complete the diary when it’s most convenient for them to do so, without much regard to the actual days in the diary. Nevertheless, the diary structure helps participants pace themselves and also gives them a clear expectation of how much content they need to produce over the course of the project. Each day in the diary covers two or three related topics and requires between 10 and 20 minutes to fill in.

We begin the diary with easy identification questions—such as asking participants to tell us about themselves and show us their phones or their homes or their clothes closet. Throughout the study, we progress through the days by bringing participants into deeper and deeper reflection on their own behaviors. We continually ask participants to both show us and explain rather than just explain. A talking head can be a very convincing way for design teams and executives to come face to face with the people they are trying to reach—particularly as they discuss real-life activities and their associated barriers and opportunities. However, an hour of just a talking head is more than monotonous; it is a story without context. We want to see how people behave in situations that are as real as we can make them.

Once we have written a protocol that we think will get people to talk about the subject matter we are studying—such as shopping behaviors across multiple devices—we pilot test it with a few people who will not take part in the actual study. We do this for several reasons. First, we want to see how long it will take participants to go through each day’s activities. Second, we want to find any parts of the text that don’t make sense to them. Because we’re so close to the topics and the research goals, our initial drafts tend to be stilted or convoluted in places. Finally, we want to see whether the questions we’re asking really encourage people to give us the types of answers we are looking for—or instead just lead to lots of blah blah blah without any real substance.

Writing a good video diary protocol is tricky. It’s very important to build in flexibility, while at the same time ensuring that participants encounter key questions at a pace that encourages them to be as thorough as possible. 

After printing out the video diary guides in full color, we give them to participants as part of a packet of materials. The packet also includes a video camera, a charger, instructions on how to use the camera, a self-addressed box with sufficient postage to return everything to us, and lots of bubble wrap and packing tape.

Selecting Video Technology

“Video continues to improve in quality, while the cost and size of video recording equipment decreases. This is a great boon to user researchers.”

 

Video continues to improve in quality, while the cost and size of video recording equipment decreases. This is a great boon to user researchers. I started out my research career with a three-chip camera, a professional tripod, multiple microphones, and video editing equipment. Now you can do all of this with a simple pocket camcorder or Webcam and free or inexpensive video editing software. The tripod I use now weighs a pound or two and folds up into a compact bundle the size of a large pencil case.

The type of camera you decide to use for consumer video diaries has multiple consequences at different stages of a project. It’s simplest to have people use Webcams that are built into their computer—unless, of course, they don’t own a computer with a Webcam. Selecting participants on the basis of the technology they own and know how to use might skew your sample in a direction that could help or harm your study, depending on whether the resulting sample reflects your target audience. If you’re designing tools for the general public, technologically savvy study participants could give you results that are not representative of the people you are trying to reach. 

In my earlier video diary studies, I tended to use Webcams exclusively. In more recent studies, I have been using Kodak Playsport cameras in VGA mode, because the resolution of the resulting videos is high enough to clearly see what’s going on, while at the same time being viewable on my iPad, which I tend to have with me everywhere. Pocket video cameras with SD cards are very easy to set up for participants, are easy for them to handle in a variety of situations, and participants can mail them back to you using readily available packing materials. I give participants a charger with the camera, just in case the batteries go dead when they’re in the middle of making their masterpiece. 

I've read about studies in which a research team distributed smartphones with cameras to participants. They used the phone to probe, calling or texting participants at particular times and asking them to take pictures or shoot video of the context they were in at that moment. This is an interesting approach, but doesn't really fit the multichannel shopping studies my team and I typically conduct.

Analyzing Video Data

“Analyzing video data is a time-intensive process, no matter how you do it. We usually watch the videos completely through once, identifying clips that give us the greatest insights into the design problem we are studying.”

 

Analyzing video data is a time-intensive process, no matter how you do it. We usually watch the videos completely through once, identifying clips that give us the greatest insights into the design problem we are studying. We write down the time code for each clip and a quick summary of the reason why we thought the clip was meaningful. Then, we go through video notes or transcripts and tag individual statements and paragraphs with descriptive words. This process is called coding the videos. There are sophisticated software programs for this process, but I don’t use them because I want the team to focus intensely on what they’re seeing and hearing and process it internally rather than focusing on getting the software to do their bidding.

We typically use an open coding scheme for the first pass or the first set of videos that we code, so we don’t restrict what tags or codes the team can use during the initial coding process. But once two or three different researchers have coded several videos, and we have a widely applicable set of tags to work with, we analyze the tags and eliminate duplicate and similar terms, so only one tag represents each unique concept.

We then create a table comprising all of our notes for each segment of a video, with participant numbers in the leftmost column, followed by a sentence or paragraph that we’ve transcribed from the videos, then the tag describing that video segment. We do this for all of the video notes or transcripts. We then sort the table by the tags, bringing together sections of the videos dealing with the same topic in the table, regardless of whose video we took them from. As we read through these similar comments, concepts and themes begin to emerge that directly address the research questions that led us to conduct the study. We refine these themes and concepts, then look for the video clips that illustrate them most succinctly. The themes translate directly into our findings. 

While creating the findings, we identify direct, verbatim quotations that we’ll later call out in the study deliverable. We also create video clips that highlight, support, and clarify our findings. Video is still a medium that people pay close attention to—albeit for a very brief time—as they evaluate whether you’ve got the goods. I have often been in meetings where everybody was staring at their own screens—on some kind of device—during our verbal presentation, but put down their devices for the first five minutes of customer videos, paying rapt attention to them. Don’t lose the moment by showing lots of mildly interesting clips that make stakeholders wonder when they will end. Instead, make sure the clips punch hard and get to the point you want to make quickly.

We do much of our video editing in Final Cut Pro, but QuickTime Pro is a very handy tool for creating highlight clips. You can trim a large clip down to a single meaningful segment, then save the segment as a new video file in a matter of seconds. Final Cut Pro requires rendering and exporting, which you must do in real time, which is very time consuming.

Formulating Recommendations

“Research findings, by themselves, are interesting, but relatively worthless in the context of a UX strategy and design research program. What matters is what you recommend as a result of the findings.”

 

Once we have a set of compelling findings, we ask the question every client will inevitably ask when we present our results: So what? This is where experienced user researchers who have a design background can provide perspectives that others cannot. Research findings, by themselves, are interesting, but relatively worthless in the context of a UX strategy and design research program. What matters is what you recommend as a result of the findings.

Writing meaningful recommendations from video diaries requires you to dig deep into the data, but also to have a general understanding of current trends and UX design patterns. Written recommendations may be enough, particularly if that is the expectation of your project sponsor. However, you may need to supplement your written recommendations with concept wireframes illustrating your recommendations. As a UX Strategy consulting agency, one of our key deliverables is a UX road map for a program that extends one to three years into the future.

Quantification

The participant sample for video diaries is relatively small in comparison to a product’s target audience. It is impossible to take data from a small sample of people, then infer any percentages that are representative of the larger population. Therefore, it is important to follow up your video diary study by proposing an approach to quantifying your findings. For example, if you have recommended a strategy that is based on a specific set of interactions across devices, you might propose an analytics measurement schema that tracks the current traffic for functionality that supports that type of interaction pattern.

Summary

“Video diaries provide a rich set of data that we can use to guide complex design programs.”

Video diaries give both UX teams and executives an up-close and personal view of the people and activities they are designing systems to support. It is a particularly useful method for gaining an in-depth understanding of the rapidly evolving ways that consumers use multiple, related technologies to achieve their goals. Multichannel UX strategy is a moving target that will continue to accelerate, but video diaries provide a rich set of data that we can use to guide complex design programs.