31 December 2013
Usability testing has become standard fare for most serious web and software development efforts over the last decade or two. The overall intent of testing is to reduce the risk of finding usability errors after product is launched. The typical “over-the-shoulder” method has served this purpose well. With this, stakeholders get a well-prepared report with a prioritized list of issues and a wealth of recommendations. All good and fine.
An alternative approach is Rapid Iterative Testing and Evaluation (RITE). This is also a lab-based method, but with an important difference to typical tests: the prototype is iteratively evaluated and updated between session. So, you not only identify problems but also test the proposed solutions.
This method was formalized about a decade ago by researchers at Microsoft, most notably Dennis Wixon. In their original paper on the approach, the researchers focus on the key benefit of improving product design:
The goals of the RITE method are to identify and fix as many issues as possible and to verify the effectiveness of these fixes in the shortest possible time. These goals support the business reason for usability testing, i.e. improving the final product as quickly and efficiently as possible. The results presented here suggest that at least in the context of its use for the Age of Empires II tutorial, the RITE method was successful in achieving its goals.
What’s more, RITE is fast: the method compresses testing, problem identification and design fixes into a short period. This is good argument to make on any project. Topics such as “Agile Design“ and “Lean UX“ are all the rage these days. With these approaches, designers seek to prototype, test and revise their designs quickly and with little documentation. RITE fits into this canon.
But there’s an additional key benefit that’s not so immediately noticeable: team engagement. RITE tests bring team members and stakeholders together. They then collectively solve design-related problems in real time — right in the observation room. This does NOT mean “design-by-committee,” where designers can easily get out-voted. On the contrary: putting designers, product managers and business stakeholders in the same room with the same stimuli gives designers a stronger voice in shaping the solution.
There are several advantages to this type of heightened team engagement:
- Common language: Collaboration during RITE tests gives rise to a common language for describing design problems and their solutions. Whether speaking about a element or overall flow, teams develop a way of describing things, which brings a certain efficiency to subsequent discussions.
- Shared references for decision-making: Witnessing users struggle using a product provides a shared reference. When updating the design, this common experience provides a center of gravity for decision making. This shared reference is typically more immediate and longer lasting than typical usability reports, for instance.
- User-centered: Perhaps most important, the users are the center of attention during the whole process. Stakeholders and other team members, who may not have regular contact with users, get a chance to view users first hand. This builds empathy for users throughout the team.
In my experience improving the UI design quickly is only half of the benefit of RITE. Equally important is the type of engagement we get from RITE. And ultimately, this is next frontier of UX design: getting the right decision-making processes in an organization that favor the user experience. RITE can help with that.
Inviting different types of stakeholders is important. We strive to include everyone from developers to marketing to project sponsors. Although this makes raises the potential of a “design by committee” effect – which you should guard against – I’ve found it lowers the chance of design decisions being overturned later.
So, consider RITE for projects where it’s appropriate. See the presentation Carola Weller and I gave at Euro IA in Rome 2012 for more:
- “Getting Software RITE,” by Jeff Patton (2008)
- “The Power of Doing It RITE,” by Nick Leggett
- “RITE,” presentation by Dennis Wixon et al. on Slideshare (2007)
- “Evaluating Usability Methods,” by Dennis Wixon in interactions (2003) [password/fee required]
 Medlock, M.C., Wixon, D., Terrano, M., Romero, R., and Fulton, B. (2002). Using the RITE method to improve products: A definition and a case study. Presented at the Usability Professionsals Association 2002, Orlando Florida.
20 July 2007
On the surface of things it looks to be quite good. I only tried out the online demo so far–the participant’s view of a sorting exercise–and it was very smooth. It’s Flash based with drag-and-drop interaction. Easy to move items around and label boxes and stuff like that.
Once the screen started getting full, however, some of the categories were cut off towards the bottom (on my computer at 1024×768), and it became difficult to move items in and out of those boxes. Otherwise, it’s simple enough for any participant to get the hang of in a matter of seconds. Both open and closed sorts are possible.
In general, one of the advantages of card sort programs is that they can help with analysis. Yet, most of the programs available overcomplicate results with dendograms and clustering and not-so intuitive statistical formats. Though I didn’t see the results from Optimal Sort in the demo, it looks to be cleaner and more straight forward from the screenshots and descriptions.
I particularly liked this claim on their site: “use common sense and experience to spot patterns.” Imagine that: you have to use your intuition as a designer to interpret the results.
The process of administering tests seems to be easy as well. In a SurveyMonkey-like fashion, you can send a link out to participants, and they can then complete the sort on the web. Of course, this leaves out what I consider an important part of card sorts: talking to people about why they grouped things together and observing where they have difficulty grouping things. But the ease of getting people to take part and efficiency this tool offers makes it valuable.
Sam Ng from Optimal Usability tipped me off to this. Thanks, Sam.
They’re still offering free sign registration with the beta version until August. Check it out.
18 July 2007
I must say that I agree with Todd’s take on usability. Sure, it’s important but it’s really a baseline for any functional design. Since everyone ultimately wants to be usable, there’s nothing usability people aren’t doing or saying that competitors aren’t either. It’s not a differentiator.
It kinda reminds me of Michael Porter’s take on corporate strategy. He essentially says that operational effeciency is important, but not strategic. It’s operational. It’s something everyone strives for, even your competitors. Can’t find the full text of Michael Porter’s “What is Strategy?” article free online, so here’s the summary and order form for the article on HBR.
“Michael Porter argues that operational effectiveness, although necessary to superior performance, is not sufficient, because its techniques are easy to imitate. In contrast, the essence of strategy is choosing a unique and valuable position rooted in systems of activities that are much more difficult to match.”