Sample Optimal Usability Newsletter
This is an example of the format and content of our monthly newsletters. If you’re interested in joining, please subscribe at the bottom of the page.
Optimal Usability Newsletter - June, 2009
In this issue:
- Tree testing: evaluating your site’s organisation
- What we’ve been doing
- Cool sites
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tree testing: evaluating your site’s organisation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Suppose you’re designing (or redesigning) the overall structure of a website. You’ve come up with a new structure - depending on the coffee, you may have come up with two or three. But are they good? Which one is best? And are they better than the old structure? Until recently, there wasn’t a quick way to find out, short of building the site and testing it after the fact.
We weren’t the only ones looking into how to test the organisation of websites. Some years ago, Donna Spencer pioneered a simple technique with a complicated name: Card-Based Classification Evaluation. Donna did her testing on paper, but it struck us that an online tool would make the technique much quicker and easier.
So we built Treejack. Its purpose was to make this “tree testing” automated and easy, while answering several key questions:
- Can users successfully find items in the site?
- Can they find those items directly, without having to backtrack?
- Can they choose between topics quickly, without having to think too much?
- Overall, which parts of the site structure work well, and which fall down?
Treejack - a tree-testing tool
Tree testing is a simple technique, so we made Treejack a simple, focused tool. Its sole purpose is to improve site organisation through quick, iterative testing. In a typical tree test, about 50 participants click a web link to start a Treejack session:
- Each participant is shown a website task (e.g. “Sign up for the company newsletter.”)
- They see a list of the site’s top-level topics.
- They click a topic, and see the subtopics under it.
- They continue clicking down until they find the answer. They can also back up and try a different path, or give up and move on to the next task.
- After about 10 tasks, they’re done. Elapsed time: 10-15 minutes.
Interpreting the results
After those 50 people have tried 10 tasks each, clicking through your site structure, Treejack can show you how well the structure performed, and more importantly, show you which parts failed with users. It summarises the results into three values:
- Success - The percentage of users who found the correct answer.
- Speed - how fast users clicked through the tree. Quick choices suggest high confidence, while hesitation suggests that topics are not clear enough.
- Directness - how directly users reached an answer. Ideally, users reach their destination without having to backtrack and try a different path.
These are measured for each task, with an overall score for the whole test. In a way, it’s like analytics for a website you haven’t built yet.
Lessons learned
We’ve run several tree tests now for large clients, and along the way, we’ve learned a few things:
- Test a few different alternatives. - Because tree tests are quick to do, we can take several proposed structures and test them against each other. This is a quick way of resolving opinion-based debates over which is better. In a recent government project, one proposed structure showed much lower success rates than the others, so we were able to discard it without regrets or doubts.
- Test new against old. - You don’t just want your new site structure to be different, you want it to be better. Tree testing is a great way to determine this. In the same government project, the original structure only had a 31% success rate. Using the same tasks, the new structure scored 67% - a solid quantitative improvement.
- Lather, rinse, repeat. - Everyone talks about being agile and iterative, but schedules and budgets often quash that ideal. Tree testing, however, has proved quick enough that we’re able to do two or three revision cycles for a given tree, using each set of results to progressively tweak and improve it.
Summary
Tree testing has become an important part of our information architecture work with clients. And Treejack has given us the automation we were after - a quick, simple-to-use tool that generates clear results quickly.
Like user testing, tree testing shows us (and our clients) where we need to focus our efforts, and injects some user-based data into the site-design process. Its simplicity means that we can do variations and iterations until we get what we came for - a tested, effective site structure.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
What we’ve been doing
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Trent, Susan, Sam and Steve just got back from a very full-on and fun Usability Professionals’ Association conference in Portland, Oregon. We’ll share some of our learnings in our next newsletter (including whether putting a mirror alongside your computer monitor makes you more productive). In the mean time take a look at Twitter to hear what people said.
On the eve of UPA, we also launched a fantastic new Optimal Workshop website and started charging for our two new products - Treejack and Chalkmark. Mad props to Steve, Sam, Andrew and William, and thanks to the hundreds of customers who have given us such valuable feedback.
Finally, congratulations to the team at Consumer, who have just launched their brand new website, and to The Warehouse, whose new e-commerce functionality has gone live for books, DVDs, CDs and games.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Cool sites
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
UXFind is a “search engine tailored for User Experience professionals”.
This letter to American Airlines and its reply makes for interesting reading about trying to do customer-centred design in a large organisation.
I can’t believe these are real banks (thanks Zef, Dmitry and Mike).
Passive Aggressive notes are funny (thanks Steve).
Don’t forget, in case of fire, do not use Twitter (thanks Susan).
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Until next time,
Warm regards,
Trent Mankelow
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Please send any suggestions, comments or questions to mailto:trent.mankelow@optimalusability.com
