Usability Testing

Assignment

As the advocate for the user, the technical communicator is responsible for anticipating and accommodating the level, needs, and assumptions of the reader. But because the technical communicator occupies a hybrid position - not quite expert, not quite public - it can be difficult to know when this accommodation is successful. In order to discover and correct problems of expression and interpretation, then, the technical communicator can design and run a user-test.

In this assignment, you will work in groups of four. Your group will choose one person's instructions to test for the user-test. The group will define success criteria, design a test, run the test on some subjects, analyze the data collected and write up a report analyzing the document's strengths and weaknesses.

Requirements of the test

What to test for. Your goal is to see if the instructions work. Can the intended readers understand them and carry them out successfully? To evaluate success, your group must set some reasonable standards of successful performance. For example, you may decide that the instructions are successful if 80% of the readers can follow the instructions in less than 10 minutes, making no more than two minor mistakes. The details of the standards of success will depend upon the instructions you test.

Who to test. While everyone will test the instructions on 8-12 real readers, your group may use either of two kinds of readers / user groups:

1) Compare the instructions against a previous set of instructions. Which one is more effective? In what ways does each set of instructions succeed? How do they fall short? Which one (if either) would you recommend as the set to distribute to the waiting public? Test at least four people on the original instructions and at least four on the new set. OR 2) Evaluate a set of instructions against two different user populations. Assuming that the instructions were written for a particular user community (e.g. math majors or anxious car owners), see how well readers from this group can use the instructions as compared to readers outside the community (e.g. English majors or budding car mechanics). Test at least 5 intended readers and at least 3 unintended readers.

Note: All group members must help to find readers and plan and conduct the tests.

How to conduct a user-test. The basic technique is to observe a reader trying to follow the instructions-without offering any assistance. You may employ any and all of the following techniques to measure your readers' success:

• time them on individual steps and overall time on task;
• record their success or failure at completing individual steps;
• take notes on their comments or problems as they read and follow the instructions;
• give them a questionnaire about how they liked the instructions.

The method you choose should reflect the criteria you set out at the beginning. Remember, whether subjects like or feel comfortable with a set of instructions is only one aspect of success, even if an important one.

What attitude to take. Take a professional attitude toward the test, whether you wrote the instructions or not. Your goal as a group is to find the best instructions for performing the task - not to judge the writer's ability to write or the reader's mental agility. Remember that the reader is always right. A "careless" mistake may be due to information that isn't as easy to spot as it should be.

Requirements for the User-Test Report

Rhetorical situation. Assume that your group is a team of document design consultants that has been hired by some organization to evaluate the instructions and recommend whether they are good enough to distribute to the public. Invent any details necessary to fill out this scenario. For example, you may assume that employees are accidentally destroying data due to faulty instructions for performing some task and that the company hired you to revise the instructions and show that they are now successful. Or your team may have been asked to write the instructions for a new commercial product that the company is planning to market. In any case, your report must make some recommendation about what the company should do with the instructions you evaluated: (1) go ahead and distribute them, or (2) do more revision and testing.

Topics to address. Your goal is to present a professional report that is both informative and persuasive. It should inform your reader (both present and potential) of your activities. It should also persuade your readers that you conducted a responsible investigation, that your conclusions are valid and that your recommendations are worth following. Remember that reports are saved as documentary evidence; you are also writing to an implicit audience of consumers, regulators, and future administrators.

Your report should follow the conventions of formal business reports and should include a title page, an abstract/executive summary, a table of contents, a table of figures and graphs, a clear organization forecasted with obvious headers, and appendices. Within the body of the report, you should review the purpose of the instructions, state the objectives of the study, describe the testing methods used, analyze the results and use the analysis to support your recommendations. Each of these requirements will be further discussed in class.

Remember, you have to live with your results, however messy or unpopular they are. The user-test is not a final evaluation but a tool for improving a company's documentation.

Schedule

Nov 3

Nov 4 Project 3 Due to Sandy by 6pm.

Nov 8 Practicing User Testing. Collaboration. Genres of report. Read Ornatowski in PWR. Notes

Nov 10 Designing the user-test interface. Interviewing subjects.

Nov 15 Gathering data. Parts/genre of the report. Read Chap 1-2 in ETC.

Nov 17 Groupwork. Bring 1-2 document to class with draft group collaboration guidelines, user-test interface, and list of subjects.

Nov 22 Thanksgiving

Nov 24 Thanksgiving

Nov 29 Visual aids: tables, charts, and graphs. Read Chaps 4-7 in ETC.

Dec 1 Workshop. Format and genre of the report.

Dec 6 Workshop.

Dec 8 Conclusions.

Dec 13 Project 4 and Portfolio Due to Sandy by 6pm.