10 Tips for Mobile Usability Testing

Go to article

By Trent Mankelow

The first time we conducted usability testing on a mobile device was in 2005 for Vodafone. The technology we used to run the testing was primitive, complicated and fragile. Here are a couple of photos to show you what the setup was like.

These days, our approach to mobile usability testing is very different. The whole process is much more guerrilla and lightweight, and we rarely test indoors. Given our recent growth in mobile testing projects, we thought we’d take a step back and share with you some of the lessons we’ve learnt.

Tip 1: Start with low-fidelity designs. We’ve known since the 80s that testing with paper-based designs is the best way to identify interface problems early on. Starting on paper de-risks mobile design projects too. For example, last year we were asked to test an early-stage mobile application that only had five simple screens. We printed out the designs, stuck to them to cardboard and took them to the street. After testing with 25 people over two days we found 25 issues, all before a line of code had been written. So our most important tip is to start mobile usability testing early, with whatever designs you’ve got. Don’t think you have to have everything finished.

Tip 2: Test in a crowded public place. Once you have a handful of screen concepts, you need to get them in front of people. By testing “in the field” you’ll find issues with noise, light and social situations that you wouldn’t otherwise uncover, which can have a big impact on the final design. We’ve had the most success testing in places with good foot traffic such as food courts, malls, cinemas and busy pedestrian streets. Before you begin you may need to ring the venue to find out what their requirements are and to get the proper clearance. For example, Westfield required that we set up a table rather than wander around their mall.

Tip 3: Use chocolate as a lure. We’ve found that a simple, universally loved treat is a good way to encourage people to help out. We generally use large blocks of chocolate, but must admit that testing during Wellington’s first snowfall in 30 years made us think that a hot chocolate voucher would have been smarter.

Tip 4: Your first sentence has to win people over. We usually invite people to take part by saying something like “block of chocolate for 10 minutes of your time?” or “we’ve got this cool thing, want to have a play with it?”

Tip 5: Ask a couple of quick screener questions. Once you have someone interested you’ll want to know whether they own or regularly use a smartphone (taking a photo of their phone can be handy at this point too). Then you might have a couple of short questions depending on what you’re testing – Do you use online banking? Do you book airline tickets online? – that kind of thing. Oh, and if people don’t qualify make sure that they still walk away with something (maybe a smaller chocolate).
Tip 6: Keep the session short, ideally 5 to 10 minutes. People aren’t going to want to speak to you standing on a street corner for very long, so you’ll only be able to get them to do a couple of tasks.

Tip 7: When people say they’d do something, make sure you get them to show you. Sometimes people say that they would click, when they really mean that they would pinch or swipe or scroll.

Tip 8: Have two people run the testing. It’s impossible to take detailed notes during a test and notice all the subtle gestures at the same time. It’s much easier to separate the note taker and moderation roles. Having three people clustered around a cardboard mock-up isn’t as intimidating for participants as you might think.

Tip 9: Don’t assume that people know mobile jargon, like App Store or iCloud. We’ve found that people are generally less experienced and comfortable with the language, metaphors, and interactions relating to mobile devices than they are with the web. So it’s important to not have too many assumptions, and avoid jargon in your test tasks.

Tip 10: Test with an interactive prototype once you’ve ironed out the early interface bugs. Further down the track, consider creating an interactive prototype in order to get a deeper understanding of how people will use your app or mobile site. It only takes a couple of hours per screen to build a prototype in a tool like Axure, and it’s possible to faithfully mimic different smartphone behaviours.

In our opinion, the quickest way to a useful, usable, and elegant mobile design is to follow a user-centred design process. This process relies on observing representative users doing representative tasks while they think aloud and you look for patterns in the things that frustrate, annoy, and delight them. The difference with testing mobile apps and websites is that most of the time people don’t use their mobile phones at their desks. Reaching out to users is their natural habitat is crucial.