Michael J. Pazzani

Department Name: Information and Computer Science

Institution: University of California, Irvine

Michael J. Pazzani

Information and Computer Science

444 Computer Science Bldg.

University of California

Irvine, CA 92697-3425

Phone: (949) 824-7405

Fax : (949) 824-4035

Email: pazzani@ics.uci.edu

http://www.ics.uci.edu/~pazzani

• Award Number: 9731990
• Duration: 10/01/1998 – 9/30/2001
• Title: From Computer Data to Human Knowledge: A Cognitive Approach to Knowledge Discovery and Data Mining

Knowledge Discovery in Database; Cognitive Science; Comprehensibility of Learned Models

The research is concerned with intelligent decision aids that can be developed by data mining techniques. Experience has shown that such systems can learn accurate models, but that experts in areas where those models are used in decision aids are often reluctant to trust them because they do not use the same tests, intermediate conclusions, or abstractions that the experts have grown to trust. Experts also want models that are stable under small changes in the data being analyzed. Psychologists have uncovered numerous factors that simplify the learning, understanding, and communication of category and process information by humans. This research seeks to explore these psychological principles in light of the output of existing KDD algorithms and to develop and evaluate new KDD algorithms that will provide output that is easy for people to learn, use, and communicate to others. With the results of this research, it should be possible to make such decision aids more "human centered", so that they will be used more often and more effectively in practice.

This is a joint project between Dorrit Billman (Georgia Institute of Technology) and Michael Pazzani (University of California, Irvine). Billman and Pazzani have complementary abilities that will produce the synergy of interdisciplinary work at its best, but also a common pool of assumptions and knowledge to facilitate communication and interaction. Pazzani is a computer scientist who has done psychology experiments, while Billman is a psychologist who has done computational work. This collaboration will bring cognitive principles into the field of Knowledge Discovery and Databases.

Our activities for the project this year fall under two main themes.

1. The evaluation of systems for multiple linear regression. Multiple linear regression (e.g., Draper and Smith, 1981) is a technique for finding a linear relationship between a set of explanatory variables (x_i) and a dependent variable (y): y = b₀ + b₁x₁ + b₂x₂ + ... + b_nx_n. The coefficients, (b_i) provide an indication on the influence of an explanatory variable on the dependent variable. Such analysis might help guide future decision-making. For example, many lenders use a credit score to help determine whether to make a loan. This score is a combination of many factors such as income, debt, and past payment history which positively or negatively affect the credit risk of a borrower. However, multiple regression as used in practice can produce models that are unacceptable to experts because factors that should positively affect a decision may have negative coefficients and vice versa. Through psychological investigations, we have verified that users are sensitive to the signs of coefficients used in linear models. We have proposed and evaluated a constrained form of regression (Independent Sign Regression) that produces models that have similar predictive accuracy to multiple linear regression, but whose results are more acceptable to users.
2. The creation and deployment of intelligent agents for text filtering. This activity also seeks to use learning algorithms that produce understandable results. In one system developed (Billsus & Pazzani, 1998), the agent can explain the reasons for classifications, accept feedback on whether the reason is acceptable to the user, and revise the user’s profile based upon feedback to the explanation.

The primary result this year is the creation of a constrained form of regression that produces linear models that generalize as well as those produced by multiple linear regression (see Table 1) yet produces results that subjects are more willing to use (see Table 2).

Table 1. Predictive Mean Squared Error of Regression Routines.

Table 2. Average Subjects Ratings for Linear Equations.

Although this research project was initiated on Sept. 30, 1998, it has attracted the attention of several corporations; the PI has been invited to speak at Microsoft Research Laboratories in Redmond, WA and HNC Software in San Diego, CA.

We believe the Independent Sign Regression algorithm represents an advance that will be applicable to a wide variety of situations, particularly those in which people with little knowledge of assumptions behind data mining algorithms apply these algorithms to diverse data sets.

1. Pazzani, M. & Bay, S. (submitted). The Independent Sign Bias: Gaining Insight from Multiple Linear Regression. Annual Meeting of the Cognitive Science Society.
2. Billsus, D. and Pazzani, M. (in press). A Personal News Agent that Talks, Learns and Explains. The Third International Conference on Autonomous Agents (Agents '99), Seattle, Washington, May 1-5, 1999.
3. Intelligent Agent software; NewsDude http://www.ics.uci.edu/~dbillsus/NewsDude

The goal of data mining is to investigate algorithms for providing insight into some phenomenon by analyzing a database of examples of that phenomenon. The specific focus of our investigation is to constrain and bias algorithms for creating models of data so that these models are understandable and coherent to users of knowledge discovery and data mining systems.

Large databases are being collected in science, business, and medicine due to advances in methods for collecting, storing, and integrating data. The potential benefit of these rich information sources has scarcely been tapped and the societal effects scarcely envisioned. Not only could this provide new discovery methods in science and new decision-making tools for business, but also new bases for policy making in health or economics, new tools for medical diagnosis, new information about products for consumers, and a host of other possibilities.

In response to the availability of these very large databases, a variety of techniques have been developed and applied to recover useful information. These techniques have drawn from statistics, pattern recognition, machine learning, and neural networks to build models describing regularities in the data. The goal of this modeling is to help people understand the data by discovering predictive or descriptive models. However, to date research in data mining has not paid attention to the cognitive factors that make the resulting models coherent, credible, easy to learn, easy to use, and easy to communicate to others. Without attention to the human user, the social benefits of data mining cannot be fully realized.

We anticipate that principles of human learning and reasoning will guide the design of new data mining algorithms to produce models that are easier for users of KDD systems to understand and that properties of learning algorithms will add to the understanding of human psychological processes.