The soap opera that is user testing (or) why observers are important

Posted by Nicole Kaufmann in usability tools, user experience, user testing on October 8, 2012 | 0 comments

With every user-testing project we do, we encourage stakeholders and project-team members to come along and observe the testing sessions.

And while some are very enthusiastic on the uptake, many never make it to any of the sessions. Excuses like: “I don’t have time”, “I’ll read the report later”, “I’ll watch the recordings”, “You can debrief me on the findings later” and “User testing is your job” seem to be all too common.

Is that so bad? Well, yes, and here’s why:

Observers get the full picture. To some degree, user tests are like soap operas – you won’t get the full picture if you see only one or two episodes. People are unique and you will learn something new from every user you observe. Watching several sessions gives you the chance to uncover trends and patterns, rather than individual comments or isolated incidents.

Observers get it straight from the horse’s mouth.  A report will always be a second-hand account, somebody else’s perspective on what they observed. Watching a user test live will give you the insights from your perspective and the motivation to change things.

Work the iron while it’s hot. Having a whole group of people observe will encourage discussion between tests. Concerns, issues, and questions can be raised while they are fresh in people’s minds and solutions can be brainstormed together.

You won’t be able to stop the wedding if you are not there. Observing user tests will always raise questions and watching recordings later won’t give you the option to ask them. If you can asked them there and then, then you can discuss them with the test facilitator and they can be incorporated into the remaining test sessions.

Multiple points of view see more. Having multiple observers (with their different perspectives and different biases) often uncovers more findings and encourages people to share observations and perspectives, leading to increased discussion and collaboration within the team.

Who bothers to watch the game, when they already know the final score? Realistically, if you can’t find the motivation to attend a live user testing session, you are not likely to watch the recordings either. To sit down and watch several hours of footage, you need to be convinced that you will see something worthwhile and often you may already have read the report and seen the results.

Have a ‘secret’ agenda. You may have your own objectives and questions which user testing might be able to answer and being there will give you the chance to push for answers to these questions and lend you support in achieving these objectives.

Less writing and selling, and more solving. If you don’t have observers from the project team, you will end up having to “sell” them the findings. This might involve a report, a presso, and on-going persuasion. Get those same people observing the sessions, though, and the selling goes away (and sometimes the formal reports too) – they’ll all be galvanised into solving the problems they saw with their own eyes.

So, what can you do to turn team members into observers? A few tips:

  • Say it up front (and remind them as the project progresses) that each team member must observe at least 1 session. If you can get the project manager to push this from the top down, even better.
  • Get their input on recruiting and the test script. They’re more likely to attend the sessions if they’ve been involved in the preparations.
  • Provide yummy snacks and drinks. People always laugh at this, but it’s true – attendance will go up if word gets around that you take good care of their bellies.

October 8, 2012. Posted by in usability tools, user experience, user testing.

Sign up to our email newsletter

Sign up to receive regular updates on our latest projects, research, and other news in the world of usability.

Email addresses are not sold or given to anyone.

Twitter updates