Chapter 7 and 8 Barker
This chapter was called Conducting Usability Tests. The chapter went over 3 basic types of tests:
1. tests for task performance (procedures)
2. tests for skill and understanding (tutorials)
3. tests for access to information (references)
2. tests for skill and understanding (tutorials)
3. tests for access to information (references)
- Hide quoted text -
The first step is deciding when to test. You can usually test at about 3 stages-- during design, during writing and development, and after the document set goes to the customer. Then you must select the test point. A test point is an issue or feature of a document that might interfere with the efficient and effective application of a program to a user's work activities. Then to select procedures for testing, you identify what issues you want to test by determining a point in your documentation where there is a chance for user error. To select design strategies for testing, look at terminology (language), an index (for consistency), cuing patterns, headlings/layout, navigation, and extraordinary document formats (special conditions). Next, you must choose the type of test to use. Relating to test points of tasks, terminology and document design strategies, you can use the Can They Do It Test, Can They Understand It Test, and the Can They Find It test. You may have to use more than one test.
Setting performance and learning objectives is important because you want the tests you perform to measure real behavior. Good performance does not necessarily mean getting something done in a short period of time. Kinds of performance objectives include
* time-related - time taken to perform a procedure
* error-related - Number of errors made during a procedure, number of times a passage gets reread before comprehending, and number of tries in the index.
Objectivity in testing means setting up the test in an unbiased way as to ensure the test does not just pass automatically to let you get on with the project. You cannot let personal bias to get into your test.
Selecting testers and evaluators is the next step. The tester is the person who administers the test: arranges the meeting with users, sets up the test situation, records the test activities, and so on. The evaluator is the person taking the usability test. Then, prepare test materials and know how you want to use them. There are many kinds of test materials, including:
* Evaluator Selection Survey - A brief list of questions that potential evaluators fill out to ensure they meet the profile as typical users.
* Instructions for Evaluators - A one-page list of instructions to help evaluators understand their role.
* Test Schedule - Schedule of testing times and locations for tester and evaluators to follow.
* Instructions for Testers - Set of instructions telling the test monitors how to conduct the test and how to record information.
* Test Lab Procedures - Set of instructions for operation the testing facility.
* Pilot Tests - Mock-up versions of test forms to try out as a way of determining how well the test works and the suitability of the performance objectives
* Test Forms - Actual tests written up as test scenarios, task descriptions, and/or procedures.
* Test Subject Materials - Actual copies of documentation that the test intends to evaluate
* Software Overviews - Marketing or overview information that informs evaluators or the main uses and features of the documentation they will use.
Kind of hardware and software test materials include a tape recorder, notebooks, camera, intercom, tables/chairs, and timing clock. All these items are used to test and simulate the actual working environment. Setting up the test environment can be a difficult task, as there is no perfect place to do this. The user's workplace, a lab, or a combination of the two. The latter two are the more expensive choices, but the user's workplace is intrusive.
Make sure information is recorded accurately. This way, you can evaluate and interpret the data to come out with a solution that best fits the user.
Chapter 7 - Writing Software Documentation - Barker
This chapter discusses getting useful reviews. To begin, Barker says a reviewer guidelines transmittal letter should accompany the document. This sheet gives document background, kind of information writer needs from reviewer, where to send mark-up copy, clear instructions for mark-up and encourages partnership with writer.
Barker details the following as guidelines for managing document reviews:
* Review the document objectives from the document plan.
* Determine the type of review needed.
* Establish a review schedule.
* Plan the review.
* Write a cover letter with questions for reviewer.
* Prepare feedback material for reviewer.
There are different types of reviews that require different people to be involved who will have different concerns regarding the review. For example, a managerial review involves managers, supervisors and team leaders who are concerned with staying budget, meeting objectives and quality control. All reviews can offer challenges and problems as all are not on-site and may not be familiar with the project. By establishing a review schedule, some problems may be eliminated.
Barker also discusses the advantages of sequential circulation verses simultaneous circulation. The writer would need to figure out which works best. The cover letter provides the following objectives and benefits to the reviewer: request for specific advice and comments, necessary background, tell reviewers how to mark or comment, give date and place for return, thank reviewers. The writer should also provide feedback materials.
Barker distinguishes between reviewing, testing and editing. Reviews require comments from several people whereas testing concentrates on users and issues of accuracy and statistics, and editing comes from trained editors who perform various levels and types of editing to your document. Barker states that reviewing is critical throughout the documentation process stages; planning and design stage, user analysis stage, development and writing stage and the draft stage. It is important to remember that all the document changes should be negotiated diplomatically. The different cultures from within a company may bring different perspectives as to how the document should look.
To conclude, Barker suggests a user walkthrough in which you present the document from beginning to end and focus questions on the key issues of usability. This review will contribute the most to helping meet the task needs. During a user walkthrough the information illustrated can only come from actual users.
Examples of questions are:
* Does the document reflect recognizable workplace activities?
* What tasks does the document not focus on that users would find important?
posted by Jaya Narayana | 8:28 PM
11 Comments:
Testing theory sounds great. I've never done it. I'm usually on to the next project once I finish a job. This all sounds idyllic. The reality is that we are all production workers who are judged according to how many widgets (documents) we make. The economic reality for all of this testing doesn't support this kind of testing the majority of the time.
I agree here with what Carl has said. I have never made a widget but I think that if all of this testing were done on every document that was created it would be very spendy and time consuming. I do think that there should be some testing to a certain extent but I think of it like getting your tires rotated on your car, if you don't rotate them it will still run. It's kind of like you need to pick and choose what tests are important and when they should be applied to certain projects.
I like to throw out the suggestion, "Let's do user testing" and I know full well it's not going to get done. We are redoing the format of our intranet. There are changes some of us don't agree on, so I suggested that we take it to the end users to see what their preference would be in an actual work-related scenario. Not. The final decision was based what was presented to our Executive Vice Presidents (many of whom don't use the intanet). It was then suggested that we could ask the users in 6 months to see how it's working. Sure...after they've gotten accustomed to the new layout.
After reading some of the comments by my peers, I’m a little torn. One on the one hand, I do understand that testing cannot occur on every document that we make. We do have deadlines and we are expected to meet them. However, when the situation calls for it, testing must occur. If we are writing a document that will be with the widgets for the next few years, that will guide the user experience with the widget, then we need testing! If this document will affect our long-term success, then we need to make time and resources available. If not, then we might as well just pack up our keyboards and move on.
I agree that we have to pick and choose what we test for usability. Testing should be saved for the really important things; documents or widgets that could cause massive backlashes if they fail. You might not test drive a bike before you buy it, but wouldn't you test drive a car first?
I find getting reviews to require a lot of direction first. As a creative writing major (this was a few years ago), I had a hard time getting people to provide useful comments on my work. Sometimes people get reviewing mixed up with copyediting. After some time being frustrated with the problem of bad reviews, I found I had better luck when I attached notes to my work asking specific questions, such as "does this part convey this emotion" or "can I do A without sacrificing B?" In my experience, the more guidance you can provide your reviewers before they look at your work, the better. Give them a history report of the document, a list of questions, or a list of what you'd like to see them do.
I think testing can be very important for large projects. It may be time-consuming and costly, but it could save time and money in the long run. In my other tech. comm. course, Writing in the Health Profession, we've recently talked about testing health campaigns at the beginning stages of development. While this was not talking specifically about usability tests, it was talking about having users test documents and materials for effectiveness. It makes sense to at least have target users review documents to see whether or not they are effective, even if you can't afford to do a full-on usability test.
In a perfect world we would test our work at every step. Alas, a perfect world doesn't exist--especially when the almighty dollar is the driving force.
I have actually done some usability testing, and my planned thesis will involve usability testing. Au contrare, usability testing IS done at some locations with some projects. There is a very active SIG in the Twin Cities that is concerned with this topic. As a few of my peers have already pointed out, selectivity is the key to usability testing.
The real question is "What (exactly) do you want to test?" Time and errors are two criteria that are suggested for testing, but I would offer that a combination of both elements would be the most useful measure. Objectivity can certainly be an issue, because you probably really want it (whatever you're testing) to work. Where to test is also an extremely important consideration because the very presence of the person giving the test can skew the results. (People generally want to please "do what is expected.")
In the tests that I conducted, the instructions were revised after the testing was completed because there were some "glitches" in those instructions.
Good job with the information in the chapter. I got a good feel of what Barker was trying to convey through your summary.
I remember when I first started in Tech Comm and I was told that once I gain knowledge in to the subject that I would never again be able to simply look at any type of document with out analyzing it for content and usability and structure. I have made widgets before as in building houses to document questionnaires. I seem to always wanting to go back and redo what I have done, from asking peoples opinions to comparison with other similar documents.
I like how logically this is lain out. The only thing I'd like to note is that it seems to be imperative that you work with someone else so that you can check each other for permitting bias to enter the tests. Of course, if you both share the same bias, this becomes an issue.
I have had to do some basic usability testing stuff for classes and it was very helpful and would be ideal for every company. I totally see why it is an expense that companies don't want to absorb though. They like to think we are so GOOD that testing will bring back no results. In reality every bit helps and most of the time some one can be used as a guinea pig.
Post a Comment
<< Home