Concepts: Usability Testing

Topics

Ways of Exposing the Design
Benefits of Exposing the Design to Various Stakeholders:

Other Project Members
External Usability Experts
Users

Usability testing evaluates the system from the perspective of the end user and includes the following types of tests:

human factors (see Concepts: User-Centered Design)
esthetics
consistency in the user interface (see Guidelines: User-Interface)
online and context-sensitive help
wizards and agents
user documentation
training materials

Usability testing is not a substitute for good design-it's most effective when combined with User-Centered Design (see Concepts: User-Centered Design).

Start usability testing early. Early user testing means early prototyping, typically drawings and mockups described as low-fidelity prototypes. The hi-fidelity prototypes follow later in the process (see Activity: Prototype the User Interface).

Ways of Exposing the Design

One way of exposing a user-interface design is to have a business or system analyst sit with the end user in front of the interface. Walk through a common scenario; for example, a use case's basic flow with typical values you described in a use-case storyboard. Encourage the person to ask questions and give comments.

The challenge with this approach is to ensure the information you obtain is as unbiased as possible. To do this, you need to make sure the questions you ask are context-free. Take as many notes as you can. If possible, have someone else do this so you don't interrupt the user's natural flow. (For useful guidelines on conducting user interviews and workshops, see Guidelines: interviews and requirements workshops.)

Another way of exposing the user-interface design is to perform use tests. These are often conducted as a lab or workshop with representatives from the end-user community. In a use test, real users perform real tasks with the interface, and the software development staff typically takes a passive, observational role.

A lot of value can be gained from this type of usability testing, however, there are a number of challenges that must be faced and tradeoffs that must be made to get reliable, economic results:

As a general rule, this approach has the most value if the end-user community is large, varied, and has a great degree of control over selecting their software system. In the presence of these factors, the risk of not performing use tests increases. Often the greater the value in performing these tests, the harder it is to gain access to, coordinate, and manage this activity with the end user.
It's important to identify the most common usage patterns, discounting outlier and exceptional results, to ensure that the user-interface design decisions are based on the needs of the majority. To do this, you need both broad and deep sample data, which usually requires a large amount of gathering and collating effort.
Where end users must migrate from an existing legacy system to a new system, they are often concerned that the new system will provide less functionality than that provided by the old one. Unfortunately, this issue is seldom raised directly, and is often concealed by comments such as "I want the new system to look and feel exactly the way the existing system does".
Where a significant change in technology is being proposed to an end-user community, it may be necessary to provide training in the basic use of the technology before significant value will be gained from use testing. For example, legacy system users may have had no previous experience using a mouse or working with a GUI.

Each project team needs to consider these challenges against the unique project environment they are working within to arrive at the appropriate timing, method, and approach to usability testing.

Benefits of Exposing the Design to Various Stakeholders

It's very important to expose the user interface to others. As the design and implementation of the interface progresses, you expose the design to increasing numbers of reviewers, including:

other project members
external usability experts
users

To get valuable feedback, you don't always have to go through formal use tests where real users perform real tasks. An important class of user-interface defects are caused by the home blindness of the user-interface designer-anyone who wasn't involved in the user-interface design should be able to identify most of these defects.

Other Project Members

This is an underestimated way of exposing the design. It has a very fast turnaround time: project members are already familiar with the application and usually they're available for a spontaneous usability session without tremendous ceremony or formality. User-interface designers should do this continuously during the design activity to cure their own home blindness.

External Usability Experts

A good usability expert can help to reduce development effort by pointing out common usability flaws, and usually offers other perspectives on the user interface based on experience. It can be valuable to involve external usability experts early in the user-interface design work, so there's sufficient time to refactor the design to incorporate their recommendations.

Design to Users

Exposing prototypes to users is generally a good use of your time. Because access to users is often limited, it's worth getting feedback on prototypes when the opportunity arises. Do this as often as necessary to gain the stakeholders' approval and to correct any misinterpretation of the stakeholders' needs. This can occur either during requirements capture or user-interface design. Wherever possible, avoid exposing the same user to the interface more than once-the second time, the user will be tainted by your earlier design ideas (similar to home blindness), and, as such, the value of the activity is diminished.

Also, when you expose a software prototype to end users, be careful to set the expectations correctly. If you don't, users may have expectations that they'll experience the full behavior of the functioning system behind the user interface.