Information and Computer Science — UC Irvine — David G. Kay — Informatics 131

FOURTH HOMEWORK (MINI-PROJECT)

Summary: Evaluate the usability of a website or application chosen from the alternatives below and propose a validated redesign of the site or application. The alternatives are

• The ICS Undergraduate Education site, www.ics.uci.edu/ugrad/. This site serves as the main source of academic information about the School of ICS at UCI.


• UCI's WebReg, www.reg.uci.edu/webreg (possibly including parts of WebSoc, http://websoc.reg.uci.edu/).


• Customer-operated checkout stations (as deployed at many large grocery stores and other retail establishments)

If you have a fourth alternative you'd like to work on, check with us right away.

This assignment has three phases: Evaluate the existing site or application, redesign it to improve the shortcomings you identified (including user tests of the new design), and propose the new design to the individual or organization in charge. Be sure to read the entire assignment right away so you're aware of everything that's involved.

You will work on this assignment in groups of three or four. We estimate that the increase in coordination and communication necessary in a four-person group roughly equals the decrease in individual workload over a three-person group and we expect that the products of three-person groups will be at least as complete and thorough as the results of four-person groups. In forming your groups, it would be wise to compare schedules, other obligations, and level of commitment to the class; your work will go most easily if all group members are roughly compatible in these respects. All members of the group will receive the same score (unless there are circumstances more extraordinary than any I've run into in the past).

Each of the alternatives poses its own challenges, which your group needs to handle as best it can. WebReg, for example, is a "live" system; you can only test aspects of it that you're actually authorized to use. The checkout stations are also "live"; it would probably be wise to observe their operation only when the store isn't busy and only after getting permission from the store management (imagine how you'd feel if you were a customer with a bunch of people hanging around the checkout station watching you and taking notes).

As in past assignments, when we give page counts here we're referring to single-spaced text in 10- or 12-point type with one-inch margins. We encourage helpful illustrations, but illustrations do not count towards the page limits.

Part I: Evaluate the existing site

• Determine three or four typical tasks that a user might perform on the site or with the application (e.g., find out when you can make an appointment with a counselor or whether you can take Informatics 131 on a P/NP basis). Choose tasks that are realistic, broadly representative of what users might want to do on the site, and substantial enough that the user can't complete them with just a few clicks in a minute or two. (The design changes you will propose in the next part should be based on the flaws you identify in this evaluation. If you already have in mind some particular changes, choose user tasks now that you think will point up problems that your contemplated changes will address.)


• Find at least three people who aren't enrolled in our class but who might be typical users of your site or application. Have each person, one at a time, do a walkthrough of the site, attempting to complete each of your three or four tasks. Follow the walkthrough guidelines discussed in class and in the book: Encourage the users to narrate what they're thinking and what problems or questions they're encountering. One team member should encourage the user to talk through the process (using non-judgemental, non-leading prompts); the other(s) should observe and take notes on the difficulties that come up. One good way to summarize the results of these walkthroughs would be to create a table with a row for each user and a column for each task; in the cells, write what happened with that user on that task: mistakes, questions, time taken, clicks or keytrokes required [not all of those, but whichever are relevant to your testing].


• For web sites, use the guidelines in the Farkas & Farkas paper (which you read as part of the previous assignment) to categorize the usability problems your users encountered. For other applications, use Farkas & Farkas where appropriate and otherwise use other guidelines from the course.


• Write a usability evaluation report of at least two and at most three pages, which could include the table mentioned above. Describe very briefly what your evaluation process was and at more length what results you found, referring to the Farkas guidelines where applicable. This will eventually be an appendix to your redesign proposal, so you should write it with an eye towards convincing the site management that you have made a thorough, methodologically sound evaluation and that the flaws you identified are more than just your personal opinion. Of course you must also write in a polite, professional tone; the site management won't take your advice if they feel it's nasty, sarcastic, or making fun of them.
  
  Try to be comprehensive, addressing most of the major flaws, even if that means overlooking minor issues or repeated instances of the same problems. (If a site includes many links that are hard to identify, for example, just say once something like, "Many links, such as the 'Policies' link on the home page, are hard to identify as links because they're not distinguished from other content. Farkas guideline 1.1 says, 'Be sure that all links indicate that they are links.' " Don't list every link that has a problem; that will fill up the available space before you have time to cover most of the important issues.) It's a good strategy to mention some successful aspects of the site, since it will make your report sound less unremittingly critical and thus more palatable to the site management, but your main goal is to propose a redesign so most of your report should address areas for improvement.

Part II: Redesign the site

• Develop a new design for the site or application, one that improves the areas you identified in your usability evaluation. The changes you propose should mainly be changes that address problems your prior user walkthroughs identified. Don't just make a bunch of changes you think would be a good idea; the point is that your redesign address the problems found in your initial user testing. Focus mainly on global issues of navigation and usability; don't spend much of your time polishing details like typefaces and graphics and the wording of the text on the pages.


• Build a prototype of your new design using Mockingbird; check with us if you think Mockingbird isn't an appropriate tool for the prototype you need to build.


• Test your prototype informally on each other during your design, making changes as necessary.


• Find at least three people who aren't in the class; they don't have to be the same people who you used before, but they may be. With each person, walk through your prototype, asking the user to perform each of the original three or four tasks and any others that you think are appropriate or necessary. As always, follow the guidelines for working with users.


• As flaws or improvements become apparent during your user testing, adjust your design and prototype and re-test the changed aspects. Keep iterating until you are satisfied with your design.

Part III: Propose your new design

• Write a proposal of three to five pages describing your redesign. The major part of the proposal should describe the design and how it improves the usability issues you identified in Part I. Of course you will include illustrations as appropriate.


• A smaller part of your proposal, not more than a page out of the maximum of five, should describe your prototyping and evaluation process and what changes resulted from that user testing. We're assuming here that the site management is interested in the process and how thorough a job you did. This part might not always reflect reality; the typical real-world proposal probably wouldn't describe the false starts and intermediate steps towards the solution. But in this case, include it.


• Share your Mockingbird prototype: Using the Share icon on Mockinbird's toolbar, get a URL for your project and paste that URL into the proposal document you're submitting. The default sharing settings should allow anyone with the URL to view your prototype.


• Submit your proposal document via Checkmate.


• Submit a PDF export of your Mockingbird site via Checkmate. If you're not using Mockingbird, submit other documentation of your prototype(s) (e.g., photos of your paper prototypes).


• Submit your usability evaluation from Part I via Checkmate.


• A realistic proposal would have a cover letter, a single simple page in business letter format whose body would say something like, "Enclosed is our proposal for the redesign of your web site. We hope you find it useful and we look forward to hearing from you." But you do not have to write or submit a cover letter for this assignment.

Part IV: Attribution

• Submit a half-page via Checkmate that describes the contributions made by each member of your team. This is not for grading purposes; except in the most extraordinary of circumstances, all team members receive the same score. But it will help us in the future (for refining the assignment or for letters of recommendation) to know who was responsible for what parts of the work.

If you would like to ask the site management what their goals and requirements are, you may do that by sending Email to i131@uci.edu. Don't pester the actual management of the site.

Written by David G. Kay, Winter 2004. Revised by David G. Kay, Summer 2007, Summer 2008, Summer 2010, Summer 2011, Summer 2012, Summer 2013.

David G. Kay, kay@uci.edu