(Originally posted April 2005 – P.S.)
There is one constant across almost all office-based work environments: the office worker is subject to innumerable interruptions and distractions. Yet when we test software and web sites for usability, we always seek to minimize interruptions and distractions. Should we?
The usability test lab environment is a contrived setting. It is purposefully designed to eliminate (or at least minimize) distractions and interruptions. It resembles nothing so much as a laboratory for psychological experiments.
Once the test session begins, the facilitator makes every effort to ensure that the participant is free from distraction. Other than asking for occasional clarification, the effective facilitator is trained to interact in a neutral, non-judgmental, clinical manner. This allows the participant to concentrate on the tasks at hand.
However, the typical office worker deals with a range of distractions and interruptions throughout the day – some self-imposed, some from external sources.
Given the ubiquitous nature of distractions and interruptions, it might make sense to replicate some of them in the test environment. Doing so in a controlled, deliberate manner would help illuminate how the product might fare under real-world conditions.
As a hypothetical example, consider an intranet site within a large corporation that allows employees to enroll in and make changes to their benefits. Imagine also that the benefits enrollment process was user tested in the usability lab with a variety of user types ranging from administrative assistants to software engineers. For our purposes, let’s assume that it earned high marks with users during this round of testing.
However, when it’s rolled out the organization finds that a significant number of employees are committing errors when enrolling or making benefits changes. Further investigation reveals that errors are most prevalent among tech support employees and mid-level managers. Why wasn’t this revealed during user testing?
One reason could be because tech support reps’ and managers’ work time is characterized by frequent interruptions and a multitude of distractions competing for their attention. While the benefits enrollment process might “test well” in the serene confines of the lab, in this case efficient use of the web site’s interface is hindered by frequent phone calls, emails, and the vagaries of working in a team environment.
It turns out to be difficult for people to ascertain the status of the application and of the operation being performed when a constant stream of distractions and interruptions characterizes their work environment. Had this been discovered before rollout, the design could have been adapted accordingly.
Studying how people use a product under conditions that replicate the distractions and interruptions of ordinary life would reveal additional information about the product that would not necessarily be revealed by traditional laboratory testing.
