Notes from November 8

Designing the Questionnaire

Avoid yes/no questions. These solicit opinion rather than quantitative response. Instead ask questions that require them to list, rank, or prioritize. It’s fine to use multiple choice but open-ended questions are often better.

For example:

• Did you think the navigation on the website was good?

Calling attention to the navigation with a question that is fishing for a "yes" answer will not give you an accurate view of the usefulness of your document.

• Better: What did you think of the navigation on the website?

The above example is at least honest, because it doesn't encourage the reader to praise the navigation. It still artificially calls attention to it.

• Best: List three good things about the website and three bad things.

Maybe the navigation will turn up in one or the other list; maybe the testers will focus their attention on different things entirely.

Many usabililty tests combine both, using a yes/no question to solicit their gut response and following it with a quantitative question.

Important: in selecting testers, be sure to remind them that

• They are volunteers.
• They can stop at any time.
• The object of inquiry is the document - not the testers, their performance or intelligence.