A Better Approach to Usability Testing

Forum One's user experience philosophy has long been "test early and often." However, we often find that usability testing is seen as a rather expansive mini-project at a particular point in the development lifecycle. Too often, we’ve found ourselves writing usability testing scripts and recruiting letters that require approval of the client before testing. The usability test sessions are usually crammed into a one- or two-day period with six to eight users. The sessions are often followed by crafting a report complete with observations and videos to convince stakeholders that some level of change is required.

This process is focused on gaining stakeholder buy-in and establishing credibility as a designer, but it does not result in the best product. It places too much importance on one activity and limits what you can learn about how a website/application meets an audience's needs.

A Better Approach
Recently we've found great value in having our Agile sprint cadence include a few hours of sharing our prototypes with users. We simply show the users whatever we have—storyboards, a clickable prototype, or functional code—and watch them interact with it while asking lots of questions.

These pre-scheduled sessions ensure that we have time built into our design and development process every two weeks to evaluate how well we are meeting users' goals, needs, and expectations. They also ensure that we are focused on improving the product, rather than adding features or spending a large amount of the project’s budget on documentation that will never be referenced once the website or application launches.

Keep It Lean
In larger usability studies, the output is usually a report documenting the findings and, perhaps, includes recommendations. In our design and development approach, however, rather than focusing on a client “deliverable,” our focus is on product improvement and iteration. We do document our findings in brief, one- to two-page memos that can be accessed by any member of the team to reflect on our design decisions.

A Few Examples
On a recent large-scale online community project, our initial research (surveys, interviews, and stakeholder conversations) suggested a very extensive feature set for events within the community. Our assumption was that events had to allow for document sharing, a robust invitation and RSVP system, and continued communication among attendees after the in-person event expired. During our prototyping phase, we realized that users actually only required an event listing and invitation portion. They could use a group feature to share documents and hold discussions. By discovering the issue in usability testing, we were able to simplify the system and offer better support to our users.

On a different project, our nonprofit client was offering consulting services for free. Naturally, we assumed this was a good thing and that it should be highlighted on the website. In fact, we emphasized these FREE services everywhere on the site. Usability testing, however, revealed that the use of the word “free” created credibility issues with members of our target audiences. Most of the participants did not want to register for the services because the “free” promise seemed too good to be true. By conducting usability tests with real content, we were able to adjust our copy-writing.

Summary
For a similar cost of a larger usability study with formalized deliverables, we’ve found that we can include several rounds of usability testing into every project. Furthermore, because we are testing the product several times, we are able to discover opportunities that would have gone unnoticed if we had only tested once.