Research in Software Explanation ­ David F. Redmiles

This research focuses on the problem of supporting programmers in using complex software libraries. It develops a cognitive theory of supporting programmers with examples and explanations. The theory is motivated by previous cognitive studies of programmers as well as real consulting experience. The theory is instantiated in a system called EXPLAINER running on a Symbolics Lisp machine. EXPLAINER relies on a semantic network representation of program examples which captures multiple knowledge perspectives. The theory is substantiated by tests of the EXPLAINER system both in a classroom setting and in a controlled empirical study.

To its users, EXPLAINER appears much like a hypermedia or computer-aided software engineering (CASE) tool. Users can view diagrammed schemas of program examples, code listings, sample execution, and text explanation. The contents of these views can be clicked on with a mouse to expand views and access additional information. However, unlike hypermedia tools, the information in the views is being generated in real time from a semantic network representation of the example. Unlike CASE tools, the diagrammed schemas and text explanation are based on knowledge perspectives about the problem the example solves and not just the structure of the example program code. Basically, in terms of cognitive theory, EXPLAINER provides its users with the "programming plans" underlying an example.

In a classroom evaluation, students learned about expert systems programming by reviewing a Mycin-like system represented in EXPLAINER. Students using EXPLAINER were able to better answer general questions about expert systems than students who reviewed the code and documentation on their own. In a more rigorous test, EXPLAINER was compared in a programming task against support provided by a hypermedia system providing conventional documentation. Programmers using EXPLAINER exhibited a more controlled and directed problem-solving process compared to those using the more conventional, though on-line, documentation. Observations in this test are corroborated by statistical analyses based on variances measured in the control groups. The result is positive in light of studies to date which have found programmers performing with great individual variability independent of the support tool provided. The success of EXPLAINER is attributed to its providing explanations based on programming plans and multiple perspective representations.

Further Information:

• A Macintosh demo of Explainer (really big file);