A facilitator sits with an individual participant, who works through a set of tasks. Observers sit in a nearby room, watching and listening through a video connection. We take notes and have a debriefing session after each participant, and again at the end of the day.
Each participant is at his or her own location, the facilitator is in another location, and the observers may be together or separate. We use software such as GoToMeeting to broadcast and record the sessions remotely, and Morae to broadcast locally.
Watch a brief video showing the observer’s view of a study.
An example of what observers see during a usability study
We’ll help you decide what to test, set up and run the study, and interpret the results with you.
What do you need to learn? Do you have paper prototypes or a finished product? Do you want to test all features or just new ones? Can participants use their own data or will we make it up?
With your input and review, we’ll prepare a task list and a set of questions to ask before and after each session. The best tasks represent what participants do in a real work situation.
We recruit participants based on personas (user categories). We can invite your customers, work with a recruiting agency or even use Craigslist to find participants. This is one of the most important parts of a study, and it can take a couple of days or a couple of weeks.
We give participants a gift for taking part in a study. It might be cash, a gift certificate or a copy of your software.
We can run a study in different locations:
Each session has one participant and lasts between 45 and 90 minutes. Through careful observation and limited interaction with the participant, we see how well the product meets each customer’s needs.
A few simple background questions give us details we use to customize the session. We give the compensation at this time to show that it doesn’t depend on particular results.
We don’t force participants to work in any order, and we adjust the tasks to make the session as realistic as possible.
We end with questions about each feature the participant used. Comparison with responses from the pre-test questions can show whether the product matched the user’s conceptual model of the task. While many questions use a rating scale, we don’t do statistical analysis of the answers; the rating scale is a way to stimulate discussion.
Your team's observations are very important. We review notes and comment with the observers at the end of each session, and again at the end of the day.
Everyone on your product team should watch some or all of the sessions to learn first hand what your customers think. It can be a very humbling experience, but it’s always educational.
Observers take notes during a session. They can send messages to the facilitator with suggestions or questions.
Using Morae or GoToMeeting lets observers watch and listen from another room or another location. One observer takes notes that are synchronized with the audio and video, making analysis of the study much easier.
The report is brief. It identifies problems and makes recommendations to fix them. We categorize and set priorities for the recommendations and meet to discuss them.
Usability reports cover these topics in addition to product-specific topics:
We can also create a short highlights video. This is particularly useful to show team members or managers who could not attend the study.