Spring 2019 — UCLA Computer Science Department — David G. Kay

FOURTH HOMEWORK (Mini-Project)

This assignment is due by 5:00 p.m. on Tuesday, June 4. It's a long one with many parts, some involving third parties and others involving unfamiliar software, so you'll need to start early and allow for the inevitable stumbling blocks. You must notify your TA by Email of your team membership; the absolute deadline for that is Friday, May 17, but that's already pretty late to get started. All the members of a team must be officially enrolled in the same discussion section. Each team must send one Email message to their TA, listing the names and IDs of each team member, with each team member cc'd on the message. Teams must get their TA's permission to change their membership after Friday, May 17; permission is not likely to be granted if the change leaves any student without membership on some team.

This assignment is worth 40% to 50% of the overall assignment score in the class. Students who do not work as part of a properly formed group will not receive full credit.

Summary: Evaluate the usability of a website or application and propose a validated redesign of the site or application.

This assignment has three phases: Evaluate the existing site or application, redesign it to improve the shortcomings you identified (including user tests of the new design), and propose the new design to the individual or organization in charge. Be sure to read the entire assignment right away so you're aware of everything that's involved.

You will work on this assignment in teams of three or four. We estimate that the increase in coordination and communication necessary in a four-person team roughly equals the decrease in individual workload over a three-person team and we expect that the products of three-person teams will be at least as complete and thorough as the results of four-person teams. In forming your teams, it would be wise to compare schedules, other obligations, and level of commitment to the class; your work will go most easily if all group members are roughly compatible in these respects. All members of the team will receive the same score (unless there are circumstances more extraordinary than any I've run into in the past); it is up to your team to consider each member's views, to work out how to resolve differences and allocate tasks fairly, and to fill in for any team member who becomes unavailable.

What about five-person teams? They are not absolutely prohibited, but we discourage them pretty strongly. Five people make it that much harder to reach group decisions, schedule meetings and user tests, get everyone to review a document, and so on. Everyone on the team receives the same score, as indicated above. Expectations of quantity and quality of work are the same for teams of three, four, or five; there's no formula for scaling a team's product according to team size. Finally, if the teams are all formed and there is one group of two people and one or more teams of five, one of the members of a five-person team will have to join the two-person group to form a three-person team. (It's best if all the students involved can agree on these membership changes and inform the TA. If the students can't agree, the TA will decide.)

Any web site or application poses its own challenges, which your group needs to handle as best it can. MyUCLA, for example, is a "live" system; you can only test aspects of it that you're actually authorized to use. Automated checkout stations in supermarkets are also "live"; it would probably be wise to observe their operation only when the store isn't busy and only after getting permission from the store management (imagine how you'd feel if you were a customer with a bunch of people hanging around the checkout station watching you and taking notes).

As in past assignments, when we give page counts here we're referring to single-spaced text in 10- or 12-point type with one-inch margins. We encourage helpful illustrations (and it's hard to get a perfect score without them), but illustrations do not count towards the page limits.

Part I: Evaluate the existing site

• Determine three or four typical tasks that a user might perform on the site or with the application (e.g., compare the laptop computers available for under $1500 or pay for purchases with a debit card). Choose tasks that are realistic, broadly representative of what users might want to do on the site, and substantial enough that the user can't complete them with just a few clicks in a minute or two. (The design changes you will propose in the next part should be based on the flaws you identify in this evaluation. If you already have in mind some particular changes, choose user tasks now that you think will point up problems that your contemplated changes will address.)
• Find at least three people who aren't enrolled in our class but who might be typical users of your site or application. Have each person, one at a time, do a walkthrough of the site, attempting to complete each of your three or four tasks. Follow the walkthrough guidelines discussed in class and in the book: Encourage the users to narrate what they're thinking and what problems or questions they're encountering. One team member should encourage the user to talk through the process (using non-judgemental, non-leading prompts); the other(s) should observe and take notes on the difficulties that come up. One good way to summarize the results of these walkthroughs would be to create a table with a row for each user and a column for each task; in the cells, write what happened with that user on that task: mistakes, questions, time taken, clicks or keytrokes required [not all of those, but whichever are relevant to your testing].
• For web sites, use the guidelines in the Farkas & Farkas paper (which you read as part of the previous assignment) to categorize the usability problems your users encountered. For other applications, use Farkas & Farkas where appropriate and otherwise use other guidelines from the course.
• Write a usability evaluation report of at least two and at most three pages, which could include the table mentioned above. Describe very briefly what your evaluation process was and at more length what results you found, referring to the Farkas guidelines where applicable. This will eventually be an appendix to your redesign proposal, so you should write it with an eye towards convincing the site management that you have made a thorough, methodologically sound evaluation and that the flaws you identified are more than just your personal opinion. Of course you must also write in a polite, professional tone; the site management won't take your advice if they feel it's careless, nasty, sarcastic, or making fun of them.
  Try to be comprehensive, addressing most of the major flaws, even if that means overlooking minor issues or repeated instances of the same problems. (If a site includes many links that are hard to identify, for example, just say once something like, "Many links, such as the 'Policies' link on the home page, are hard to identify as links because they're not distinguished from other content. Farkas guideline 1.1 says, 'Be sure that all links indicate that they are links.' " Don't list every link that has a problem; that will fill up the available space before you have time to cover most of the important issues.) It's a good strategy to mention some successful aspects of the site, since it will make your report sound less unremittingly critical and thus more palatable to the site management, but your main goal is to propose a redesign so most of your report should address areas for improvement.

Part II: Redesign the site

• Develop a new design for the site or application, one that improves the areas you identified in your usability evaluation. The changes you propose should mainly be changes that address problems your prior user walkthroughs identified. Don't just make a bunch of changes you think would be a good idea; the point is that your redesign address the problems found in your initial user testing. Focus mainly on global issues of navigation and usability; don't spend much of your time polishing details like typefaces and graphics and the wording of the text on the pages.
  
• Build a prototype of your new design using the system you used in the previous assignment, or one of the other systems listed in that assignment. You may also wish to try Axure RP, for which you can request a free student license and view tutorial videos on line.

• Test your prototype informally on each other during your design, making changes as necessary.
• Find at least three people who aren't in the class; they don't have to be the same people whom you used before, but they may be. With each person, walk through your prototype, asking the user to perform each of the original three or four tasks and any others that you think are appropriate or necessary. As always, follow the guidelines for working with users.
  
• As flaws or improvements become apparent during your user testing, adjust your design and prototype and re-test the changed aspects. Keep iterating until you are satisfied with your design.

Part III: Propose your new design

• Write a proposal of three to five pages describing your redesign. The major part of the proposal should describe the design and how it improves the usability issues you identified in Part I. Of course you will include illustrations as appropriate.
• A smaller part of your proposal, not more than a page out of the maximum of five, should describe your prototyping and evaluation process and what changes resulted from that user testing. We're assuming here that the site management is interested in the process and how thorough a job you did. This part might not always reflect reality; the typical real-world proposal probably wouldn't describe the false starts and intermediate steps towards the solution. But in this case, include it.
• Submit your proposal document via CCLE. Include your usability evaluation from Part I as an appendix. As before, only one member of the group should do the submission, including the names of all group members. The submitting member should Email this document to all the team members right after submitting. Group members who do not recieve that Email should contact the submitting member and their TA.
• Export your prototype in some form (like a web document or a PDF) that will let the reader view the design and trace the links or interactions. Submit that exported prototype via CCLE. Follow the same submitting process as above.
• A realistic proposal would have a cover letter, a single simple page in business letter format whose body would say something like, "Enclosed is our proposal for the redesign of your web site. We hope you find it useful and we look forward to hearing from you." But you do not have to write or submit a cover letter for this assignment; we're just saving you that extra hour of work since it's not that hard and not that vital to the main themes of the course.

Part IV: Attribution

• Each team must prepare jointly a document that describes the contributions made by each member of the team. One or two sentences for each member is sufficient. This is not for grading purposes; all team members receive the same score. But it will help us in the future (for refining the assignment or for letters of recommendation) to know who was responsible for what parts of the work. [We could imagine extraordinary circumstances where that could change, but we've never actually seen such circumstances occur. They certainly wouldn't include the usual conflicts that come up in any group. It's up to you to overcome those and produce your report.] Every member of the group must sign this attribution document. Then scan or photograph the signed document and submit it with the rest of your team's work via CCLE.

If you would like to ask the site management what their goals and requirements are, you may do that by sending Email to your TA. Don't pester the actual management of the site.

Written by David G. Kay, Winter 2004. Revised by David G. Kay, Summer 2007, Summer 2008, Summer 2010, Spring 2019.