ICS News Archive

Graduate Student Spotlight: Markelle Kelly Highlights the Human Side of Computer Science

Recognizing the importance of cross-disciplinary work, computer science Ph.D. student Markelle Kelly is partnering with cognitive scientists on her research into human-machine collaboration. “Collaborating with people who are really knowledgeable in this realm results in a much more useful framework than if machine learning researchers tried to reinvent the wheel ourselves,” she says, highlighting ongoing research efforts between UC Irvine’s Donald Bren School of Information and Computer Sciences (ICS) and the Department of Cognitive Sciences. In fact, her collaborative approach has helped her earn multiple fellowships at UCI. Last summer, she received a fellowship through UCI’s Steckler Center for Responsible, Ethical, and Accessible Technology (CREATE), and this year she was awarded fellowships through the Irvine Initiative in AI, Law, and Society and the Hasso Plattner Institute (HPI at UCI). Here, she talks about her machine learning research and future plans.

How did you first become interested in computer science?
Early on in my statistics undergraduate program, I was required to take a programming course. I found coding a useful and fun skill, so I fed that interest with a variety of additional computer science courses and a software engineering internship. As I continued my studies, I realized that computer science was the key to applying the theoretical statistical concepts that I was learning to the real world. By the time I applied to graduate school to research machine learning, I knew that I wanted to supplement my statistical knowledge with a computer science approach.

What led you to UCI for your Ph.D.?
UCI was a natural choice for my Ph.D.— it has a great computer science program, where a lot of really exciting machine learning research is happening. A big deciding factor was my adviser, Dr. Padhraic Smyth, who is very knowledgeable and supportive. Also, I grew up in the Seattle area, so the weather and proximity to the beach were nice bonuses!

What motivated you to focus on human-machine collaboration?
A lot of machine learning research is focused on performance improvements in terms of metrics like accuracy. This is important work, but its impact is limited by whether the people using an algorithm can understand it, trust it and use it appropriately. Thus, exploring how machine learning models are used in practice, and how we can effectively delegate tasks to them, is a critical area of research — one that will become increasingly important as these models become our decision-making partners to a larger and larger extent.

This realm of human-machine collaboration appealed to me because I really enjoy the “human” side of computer science. I’ve studied user interface design and was able to participate hands-on in the design process during my time in software engineering. It’s a lot of fun to use tools like visualizations and user studies, but also get to do the “mathy,” technical work.

Can you talk about some of your work as a graduate researcher for CREATE?
When a machine learning model makes a prediction, it is accompanied by a confidence score, an estimated probability that the prediction is correct. This score is really important for someone deciding whether to trust the model’s prediction, and unfortunately models are often over- or underconfident. We say that a model is “well-calibrated” if the confidence scores and accuracy match up closely.

During my CREATE fellowship, I developed diagnostic tools for model calibration, focusing on systematic biases in calibration. For example, a model might be underconfident for younger people and overconfident for older people. My work helps detect, characterize and mitigate these biases, using a new performance metric and visualization technique, so people can better understand and improve their models.

Why is it important to take a cross-disciplinary approach?
Machine learning touches so many other disciplines. Just with respect to human-machine collaboration, there’s a wealth of relevant work in software development, user-centered design, human-computer interaction, psychology and cognitive science. For a researcher working purely in machine learning, it might be hard to find and effectively use the right information for a project.

That’s why it has been so awesome for me to work with cognitive science researchers who are experts in their field. For example, I am currently developing a Bayesian framework for modeling humans’ mental models of machine learning agents. This will help us understand how people develop beliefs about what a model “knows,” or what its strengths and weaknesses are. There are theories and techniques for this type of work that education and cognitive science researchers have carefully designed and tested over decades; they just haven’t been applied to machine learning problems yet. Collaborating with people who are really knowledgeable in this realm results in a much more useful framework than if machine learning researchers tried to reinvent the wheel ourselves.

What are your future plans?
I plan on continuing to research and develop tools for more interactive, interpretable and accountable machine learning. After finishing my Ph.D., I would like to do research in industry. I am really interested in the real-world application of these tools, and moving outside of academia will illuminate practical motivations and use cases.

What do you like to do in your free time?
Living in Irvine makes it really easy to enjoy the outdoors: I like to go for hikes in the area and try to make it to the beach at least once a week. I also enjoy cooking, reading, doing puzzles (both crossword and jigsaw) and exploring the cool restaurants and nightlife in L.A. and San Diego.

Do you have any advice for prospective graduate students?
First of all, go for it! People often make getting your Ph.D. sound really intimidating, but I think it’s the best “job” in the world. I have lots of flexibility to work on projects that are interesting to me and great resources to support my work.

Second, learn to prioritize your work-life balance. When I was in undergrad, I was definitely guilty of racing to the finish, making school a test of endurance. I have learned, though, that I do much better work when I am sleeping enough, taking care of myself, and having fun outside of work. It makes school a lot more enjoyable too!

— Shani Murray