ICS 4 • David G. Kay • UC Irvine

Sixth Homework (Mini-Project Part III)

This assignment is due by 7:00 p.m. on Thursday, March 14.

You will continue working with the same group on the same web site.

Part (a)

By this point, you already have decided what aspect(s) of your web site to (re)design, you have come up with one or two designs (depending on whether you've taken the refine-one-design approach or the two-alternatives approach), and you have built at least one prototype of that design. The next step is to plan your user tests.

What are the tasks you're going to ask the user to do? What are you going to measure (time, questions, errors, overall satisfaction, …) and how will you make the measurements (by what combination of observation, exit interview, other means)? Once you settle on this explicitly, you may find that you need to adjust your prototype(s) to include some aspect or alternative that you overlooked earlier.

Who will be your test subjects? Choose people whose characteristics are representative of the intended users of the system. Try not to use family members or close friends, because they may not be candidly critical of the design. Aim for five test subjects for each round of testing (that's ten user tests in total, whether refine-one-design or two-alternatives); that will allow for no-shows or other failures (consider three subjects per round as an absolute minimum). For convenience, you may use the same group of five for each round, though in reality their experience with the first test may bias their performance on the second; in real life, it would be better for the second group of test subjects to be different, but evenly matched with, the first group.

During the user tests, which of your team members will be the facilitator, the computer, the observer(s), other roles? [In real user testing, nobody on the testing team would switch roles between test subjects because it would make the testing inconsistent. For purposes of this assignment, however, you may switch roles for different test subjects if you wish, just so each team member can experience more than one role.]

Part (b)

Review the slides and the textbook to reacquaint yourselves with the guidelines for running successful user walkthroughs.

If you're following the two-alternatives approach: Run your user tests on each of the two alternative design prototypes, making the measurements and observations you decided on earlier. Was one alternative the clear winner, or did you identify aspects of each alternative that you would include in the final design? Based on these tests, what will your final design be?

If you're following the refine-one-design approach: Run your user tests on your initial design prototype, making the measurements and observations you decided on earlier. What flaws or potential improvements did your tests reveal? Redesign your site based on these insights, and revise your prototype or re-create it to reflect this improved design. Then run a second round of user tests on the revised prototype, measuring and observing as before. Did your changes result in an improvement? Based on these tests, what will your final design be?

For this assignment, turn in a document that describes your (re)design effort, which should include

• an overall description of your site and a description of the aspect(s) you decided to (re)design (about half a single-spaced page of text, not counting any illustrations). You can re-use or revise what you wrote about this for the last assignment; this describes your starting point and we include it again here so that the document you produce for this assignment will be comprehensive and self-contained.
• a description of the requirements of your (re)design. You can re-use or revise what you wrote about this for Assignment Four.
• a description of your user testing plan (including whether you took the two-alternative or the refine-one-design approach): the task(s) for the test subjects and the measurements and observations you planned (at most two pages)
• a description of how the user testing went in practice: did it go according to plan, were there unforeseen difficulties, did anything surprising happen? (at most one page)
• a list describing each test subject's relevant characteristics, perhaps including age, gender, occupation, level of technical sophistication, previous experience with your web site (if it previously existed) or the subject matter of the site. To preserve your testers' privacy, it's not necessary to include the subjects' actual names. State whether you used the same test subjects for both rounds of testing.
• a description of the user testing results and how you applied them to your (re)design (at most four pages):
  • If you took the two-alternatives approach: What did you learn about each alternative? Did the results help you pick one alternative or identify positive aspects of each? Describe your final design (with illustrations as appropriate).
  • If you took the refine-one-design approach: What changes did your first round of testing cause you to make? Did those changes turn out to be an improvement in the second round of testing? How did the both rounds influence your final design? Describe your final design (with illustrations as appropriate).
• the names of all your team members along with a sentence for each team member describing his or her roles and contributions to the project. Also indicate whether your HCI notebooks were a useful resource as you did this project. [Except in extraordinary cases, this will not be used for assigning different scores to different team members.]