Usability Testing

Back to ENGL303

From http://www.usability.gov It is important to realize that usability is not a single, one-dimensional property of a user interface. Usability is a combination of factors including:

• Ease of learning - How fast can a user who has never seen the user interface before learn it sufficiently well to accomplish basic tasks?
• Efficiency of use - Once an experienced user has learned to use the system, how fast can he or she accomplish tasks?
• Memorability - If a user has used the system before, can he or she remember enough to use it effectively the next time or does the user have to start over again learning everything?
• Error frequency and severity - How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors?
• Subjective satisfaction - How much does the user like using the system?

The basic technique: observe a user trying to follow the instructions without offering any assistance. There are other techniques as well, for example: user personae, user scenarios, tests and compares different populations, and so on.

Your goal is to see how successfully the users follow the instructions on the website. Can the intended users understand them (short of carrying out the actual task)? Your test subjects will be members of other groups in the class. You must define your testing approach and what you will measure.

When to test usability?

• Before designing, test the old design;
• before designing, test alternative or competitive designs;
• during design, test prototypes, etc.;
• after design, test final version.

Typical testing techniques:

• time users on individual steps and overall time on task;
• record user's success or failure at completing individual steps;
• take notes on user's comments or problems as they read and follow the instructions ("think aloud");
• give user's a questionnaire about how they liked the instructions.

Note: Your test will go beyond simply following the instructions. For example, what happens if they go through the instructions with the screen minimized?

Testing attitude. Take a professional attitude toward testing, whether you wrote the instructions or not. Your goal as a group is to find the best instructions for performing the task. Remember that the user is always right. A "careless" mistake may be due to information that isn't as easy to spot as it should be.

Considerations for test questionnaires

Typicaly tests require subjects to fill out initial information before the test and follow-up information after the test. The initial information is typically personal data, including familiarity with the task and/or interface being tested. The follow-up information is based on the testing experience.

In general, avoid yes/no questions. These solicit opinion rather than quantitative response. Instead ask questions that require them to list, rank, or prioritize. It’s fine to use multiple choice but open-ended questions are often better. For example:

• Did you think the navigation on the website was good?

Calling attention to the navigation with a question that is fishing for a "yes" answer will not give you an accurate view of the usefulness of your document.

• Better: What did you think of the navigation on the website?

The above example is at least honest, because it doesn't encourage the reader to praise the navigation. It still artificially calls attention to it.

• Best: List three good things about the website and three bad things.

Maybe the navigation will turn up in one or the other list; maybe the testers will focus their attention on different things entirely.

Many usabililty tests combine both, using a yes/no question to solicit their gut response and following it with a quantitative question.

Important: in selecting testers, be sure to remind them that

• They are volunteers.
• They can stop at any time.
• The object of inquiry is the document - not the testers, their performance or intelligence.