Chapter 7 and 8 Barker
This chapter was called Conducting Usability Tests. The chapter went over 3 basic types of tests:
2. tests for skill and understanding (tutorials)
3. tests for access to information (references)
The first step is deciding when to test. You can usually test at about 3 stages-- during design, during writing and development, and after the document set goes to the customer. Then you must select the test point. A test point is an issue or feature of a document that might interfere with the efficient and effective application of a program to a user's work activities. Then to select procedures for testing, you identify what issues you want to test by determining a point in your documentation where there is a chance for user error. To select design strategies for testing, look at terminology (language), an index (for consistency), cuing patterns, headlings/layout, navigation, and extraordinary document formats (special conditions). Next, you must choose the type of test to use. Relating to test points of tasks, terminology and document design strategies, you can use the Can They Do It Test, Can They Understand It Test, and the Can They Find It test. You may have to use more than one test.
Setting performance and learning objectives is important because you want the tests you perform to measure real behavior. Good performance does not necessarily mean getting something done in a short period of time. Kinds of performance objectives include
* time-related - time taken to perform a procedure
* error-related - Number of errors made during a procedure, number of times a passage gets reread before comprehending, and number of tries in the index.
Objectivity in testing means setting up the test in an unbiased way as to ensure the test does not just pass automatically to let you get on with the project. You cannot let personal bias to get into your test.
Selecting testers and evaluators is the next step. The tester is the person who administers the test: arranges the meeting with users, sets up the test situation, records the test activities, and so on. The evaluator is the person taking the usability test. Then, prepare test materials and know how you want to use them. There are many kinds of test materials, including:
* Evaluator Selection Survey - A brief list of questions that potential evaluators fill out to ensure they meet the profile as typical users.
* Instructions for Evaluators - A one-page list of instructions to help evaluators understand their role.
* Test Schedule - Schedule of testing times and locations for tester and evaluators to follow.
* Instructions for Testers - Set of instructions telling the test monitors how to conduct the test and how to record information.
* Test Lab Procedures - Set of instructions for operation the testing facility.
* Pilot Tests - Mock-up versions of test forms to try out as a way of determining how well the test works and the suitability of the performance objectives
* Test Forms - Actual tests written up as test scenarios, task descriptions, and/or procedures.
* Test Subject Materials - Actual copies of documentation that the test intends to evaluate
* Software Overviews - Marketing or overview information that informs evaluators or the main uses and features of the documentation they will use.
Kind of hardware and software test materials include a tape recorder, notebooks, camera, intercom, tables/chairs, and timing clock. All these items are used to test and simulate the actual working environment. Setting up the test environment can be a difficult task, as there is no perfect place to do this. The user's workplace, a lab, or a combination of the two. The latter two are the more expensive choices, but the user's workplace is intrusive.
Make sure information is recorded accurately. This way, you can evaluate and interpret the data to come out with a solution that best fits the user.
Chapter 7 - Writing Software Documentation - Barker
This chapter discusses getting useful reviews. To begin, Barker says a reviewer guidelines transmittal letter should accompany the document. This sheet gives document background, kind of information writer needs from reviewer, where to send mark-up copy, clear instructions for mark-up and encourages partnership with writer.
Barker details the following as guidelines for managing document reviews:
* Review the document objectives from the document plan.
* Determine the type of review needed.
* Establish a review schedule.
* Plan the review.
* Write a cover letter with questions for reviewer.
* Prepare feedback material for reviewer.
There are different types of reviews that require different people to be involved who will have different concerns regarding the review. For example, a managerial review involves managers, supervisors and team leaders who are concerned with staying budget, meeting objectives and quality control. All reviews can offer challenges and problems as all are not on-site and may not be familiar with the project. By establishing a review schedule, some problems may be eliminated.
Barker also discusses the advantages of sequential circulation verses simultaneous circulation. The writer would need to figure out which works best. The cover letter provides the following objectives and benefits to the reviewer: request for specific advice and comments, necessary background, tell reviewers how to mark or comment, give date and place for return, thank reviewers. The writer should also provide feedback materials.
Barker distinguishes between reviewing, testing and editing. Reviews require comments from several people whereas testing concentrates on users and issues of accuracy and statistics, and editing comes from trained editors who perform various levels and types of editing to your document. Barker states that reviewing is critical throughout the documentation process stages; planning and design stage, user analysis stage, development and writing stage and the draft stage. It is important to remember that all the document changes should be negotiated diplomatically. The different cultures from within a company may bring different perspectives as to how the document should look.
To conclude, Barker suggests a user walkthrough in which you present the document from beginning to end and focus questions on the key issues of usability. This review will contribute the most to helping meet the task needs. During a user walkthrough the information illustrated can only come from actual users.
Examples of questions are:
* Does the document reflect recognizable workplace activities?
* What tasks does the document not focus on that users would find important?