Information and Computer Science — UC Irvine — David G. Kay — Informatics 131
FOURTH HOMEWORK (MINI-PROJECT)
This assignment is a long one with many parts, some involving third parties
and others involving unfamiliar software, so you'll need to start early
and allow for the inevitable stumbling blocks. This assignment is worth about 50% of the overall assignment score in the class.
Summary: Evaluate the usability of
a website or application chosen from the alternatives below and propose a validated
redesign of the site or application. The alternatives are
- The ICS Undergraduate Education site, www.ics.uci.edu/ugrad/. This site serves as the main source of academic information about the School of ICS at UCI.
- UCI's WebReg, www.reg.uci.edu/webreg (possibly including parts of WebSoc, http://websoc.reg.uci.edu/).
Customer-operated checkout stations (as deployed at many large grocery stores and other retail establishments)
If you have a fourth alternative you'd like to work on, check with us right away.
This assignment has three phases: Evaluate
the existing site or application, redesign it to improve the shortcomings you identified
(including user tests of the new design), and propose the new design to
the individual or organization in charge. Be sure to read the entire assignment right away so
you're aware of everything that's involved.
You will work on this assignment in groups
of three or four. We estimate that the increase in coordination and communication
necessary in a four-person group roughly equals the decrease in individual
workload over a three-person group and we expect that the products of three-person groups will be
at least as complete and thorough as the results of four-person groups. In forming your groups, it would be wise to compare schedules, other obligations, and level of commitment to the class; your work will go most easily if all group members are roughly compatible in these respects. All members of the group will receive the same score (unless there are circumstances more extraordinary than any I've run into in the past).
Each of the alternatives poses its own challenges, which your group needs to handle as best it can. WebReg, for example, is a "live" system; you can only test aspects of it that you're actually authorized to use. The checkout stations are also "live"; it would probably be wise to observe their operation only when the store isn't busy and only after getting permission from the store management (imagine how you'd feel if you were a customer with a bunch of people hanging around the checkout station watching you and taking notes).
As in past assignments, when we give page
counts here we're referring to single-spaced text in 10- or 12-point
type with one-inch margins. We encourage helpful illustrations, but illustrations
do not count towards the page limits.
Part I: Evaluate the existing site
Determine three or four typical tasks that a user might
perform on the site or with the application (e.g.,
find out when you can make an appointment with a counselor or whether you can take Informatics 131 on a P/NP basis). Choose tasks that
are realistic, broadly representative of what users might want to do on
the site, and substantial enough that the user can't complete them with
just a few clicks in a minute or two. (The design changes you will propose in the next part should be based on the flaws you identify in this evaluation. If you already have in mind some particular changes, choose user tasks now that you think will point up problems that your contemplated changes will address.)
Find at least three people who aren't enrolled
in our class but who might be typical users of your site or application. Have each person, one at a time, do a walkthrough of the
site, attempting to complete each of your three or four tasks. Follow the walkthrough
guidelines discussed in class and in the book: Encourage the users to narrate
what they're thinking and what problems or questions they're encountering.
One team member should encourage the user to talk through the process (using
non-judgemental, non-leading prompts); the other(s) should observe and take
notes on the difficulties that come up. One good way to summarize the results of these walkthroughs would be to create a table with a row for each user and a column for each task; in the cells, write what happened with that user on that task: mistakes, questions, time taken, clicks or keytrokes required [not all of those, but whichever are relevant to your testing].
For web sites, use the guidelines in the Farkas & Farkas paper
(which you read as part of the previous assignment) to categorize the usability
problems your users encountered. For other applications, use Farkas & Farkas where appropriate and otherwise use other guidelines from the course.
Write a usability evaluation report of at
least two and at most three pages, which could include the table mentioned above. Describe very briefly what your evaluation
process was and at more length what results you found, referring to the
Farkas guidelines where applicable. This will eventually
be an appendix to your redesign proposal, so you should write it with an
eye towards convincing the site management that you have made a thorough,
methodologically sound evaluation and that the flaws you identified are
more than just your personal opinion. Of course you must also write in
a polite, professional tone; the site management won't take your advice
if they feel it's nasty, sarcastic, or making fun of them.
Try to be comprehensive, addressing most of
the major flaws, even if that means overlooking minor issues or repeated
instances of the same problems. (If a site includes many links that are
hard to identify, for example, just say once something like, "Many
links, such as the 'Policies' link on the home page, are hard to
identify as links because they're not distinguished from other content.
Farkas guideline 1.1 says, 'Be sure that all links indicate that they
are links.' " Don't list every link that has a problem; that
will fill up the available space before you have time to cover most of the
important issues.) It's a good strategy to mention some successful
aspects of the site, since it will make your report sound less unremittingly
critical and thus more palatable to the site management, but your main goal
is to propose a redesign so most of your report should address areas for
Part II: Redesign the site
Develop a new design for the site or application, one that
improves the areas you identified in your usability evaluation. The changes you propose should mainly be changes that address problems your prior user walkthroughs identified. Don't just make a bunch of changes you think would be a good idea; the point is that your redesign address the problems found in your initial user testing. Focus mainly
on global issues of navigation and usability; don't spend much of your
time polishing details like typefaces and graphics and the wording of the
text on the pages.
Build a prototype of your new design using
Mockingbird; check with us if you think Mockingbird isn't an appropriate tool for the prototype you need to build.
Test your prototype informally on each other
during your design, making changes as necessary.
Find at least three people who aren't in
the class; they don't have to be the same people who you used before,
but they may be. With each person, walk through your prototype, asking
the user to perform each of the original three or four tasks and any others that
you think are appropriate or necessary. As always, follow the guidelines
for working with users.
As flaws or improvements become apparent during
your user testing, adjust your design and prototype and re-test the changed
aspects. Keep iterating until you are satisfied with your design.
Part III: Propose your new design
Write a proposal of three to five pages describing
your redesign. The major part of the proposal should describe the design
and how it improves the usability issues you identified in Part I. Of course
you will include illustrations as appropriate.
A smaller part of your proposal, not more
than a page out of the maximum of five, should describe your prototyping
and evaluation process and what changes resulted from that user testing.
We're assuming here that the site management is interested in the process
and how thorough a job you did. This part might not always reflect reality;
the typical real-world proposal probably wouldn't describe the false
starts and intermediate steps towards the solution. But in this case, include
- Share your Mockingbird prototype: Using the Share icon on Mockinbird's toolbar, get a URL for your project and paste that URL into the proposal document you're submitting. The default sharing settings should allow anyone with the URL to view your prototype.
- Submit your proposal document via Checkmate.
Submit a PDF export of your Mockingbird site via Checkmate. If you're not using Mockingbird, submit other documentation of your prototype(s) (e.g., photos of your paper prototypes).
Submit your usability evaluation from Part
I via Checkmate.
A realistic proposal would have a cover letter,
a single simple page in business letter format whose body would say something
like, "Enclosed is our proposal for the redesign of your web site.
We hope you find it useful and we look forward to hearing from you."
But you do not have to write or submit a cover letter for this assignment.
Part IV: Attribution
- Submit a half-page via Checkmate that describes the contributions made by each member of your team. This is not for grading purposes; except in the most extraordinary of circumstances, all team members receive the same score. But it will help us in the future (for refining the assignment or for letters of recommendation) to know who was responsible for what parts of the work.
If you would like to ask the site management
what their goals and requirements are, you may do that
by sending Email to firstname.lastname@example.org.
Don't pester the actual management of the site.
Written by David G. Kay, Winter 2004. Revised by David G. Kay, Summer 2007, Summer 2008, Summer 2010, Summer 2011, Summer 2012, Summer 2013.
David G. Kay, email@example.com