\chapter{Introduction}
\label{introduction}

\section{Software Uncertainty Modeling}
\label{introsection}

Software is a large and complex artifact, produced primarily
by humans for humans.  Software development and operational
use are therefore dominated by human action, human judgment
and decision making, and inevitably human error.
The outcome is, therefore, often uncertain and unpredictable
and, along with uncertainty in natural phenomena, leads to
unavoidable software uncertainties.
Most if not all software failures are experienced by or
otherwise affect human users.  Similarly, most software
defects can be traced to human origins, be it an error
introduced directly by a developer or a defect that is
only indirectly human--related.  An IEEE software
engineering standard, for example, identifies two key
sources of software uncertainty as ``poor knowledge
or understanding of the problem domain'' and
``insufficient, untimely orientation to user needs and to
overall quality''~\cite{IEEE9822}.

The observation
that ``software engineering is fraught with uncertainty''
is often used to explain the known symptoms of the
software crisis, including high costs and unexpected
slips in development schedule as well as unpredictable
failures of operational software.  The abundance of
uncertainty also justifies extensive measures taken
by researchers and practitioners attempting to identify
and alleviate software risks and uncertainties.

While software risk management has long been an active
field of research producing significant contributions
(cf.~\cite{Boehm:riskbook,Boehm:riskpaper}), the subject
of software uncertainties has largely been overlooked,
with the notable exception of software reliability
models~\cite{littlewood79} and models for validation
of software dependability~\cite{Littlewood:cacm}.
This omission is particularly surprising in light of
extensive research in modeling and management of
uncertainty carried out in Artificial Intelligence
(AI) domains (cf.~\cite{Pearl88,Neapolitan90,cacm:uncertainty}).

Thus, despite the pervasiveness of human involvement
and related software uncertainties, surprisingly few
attempts have been made to model those uncertainties
explicitly.  This dissertation aims to remedy this
situation, initially by presenting the Maxim of 
Uncertainty in Software Engineering (MUSE) and as
a corollary, urging that
software uncertainties be modeled directly, using
an appropriate uncertainty modeling technique.

Ideally, an uncertainty modeling technique should include
probabilistic notions of uncertainty and confidence,
provide for multiple sources of evidence and influence,
and support dynamic updating of uncertainty values during
software development.  In this dissertation, we claim
that Bayesian Belief Networks meet these desiderata and
are therefore suitable for software uncertainty modeling.

There are many perceived benefits of modeling software
uncertainty, including, among others:
\begin{enumerate}
\item
Uncertainty permeates software development, yet is generally
absent in most software models.  A model providing explicit
representation of software uncertainties would therefore be
more accurate (and possibly more useful) than one where
uncertainties are disregarded.
\item
The search for uncertainty plays a pivotal role in an overall
software risk management strategy.  Following
Gemmer~\cite{Gemmer:risk}, ``It is impossible to manage risk
without ferreting out elements of uncertainty.  Uncertainty
is present in all decisions we face.''  A key risk
minimization activity is to ``systematically search for
uncertainty wherever it may be.''
%uncertainty wherever it may be''~\cite{Gemmer:risk}.
\item
Finally, software uncertainty modeling may help guide and 
ultimately improve software process decisions.  Based on
changes and fluctuations in software uncertainty levels, 
for example, a project manager may decide to ``conclude
testing,'' ``instigate regression testing,'' or else
``shuffle testing priorities.''  
\end{enumerate}

We generally believe that software uncertainty modeling
holds promise for improved human understanding and decision
making in a wide range of development activities.  In this
dissertation, however, we focus on specific benefits of
uncertainty modeling for specific development situations.
To this end, we describe several premises and propose a
specific research hypothesis to be validated against a
real software project.  The premises and hypothesis for
this dissertation are described next.

\section{Premises and Hypothesis}
\label{premises}

To arrive at the approach taken in this dissertation,
one must follow the statements in this section.
Most statements, such as ``software complexity hinders
its understanding'' or ``users get disoriented in
large hypertext spaces,'' are either widely accepted
or substantiated later.  These statements therefore
constitute the {\em premises} for this dissertation.
One statement, however, is less accepted and requires
further investigation: ``Explicit modeling of
software uncertainties improves human judgment and
decision making during software development.''
This statement therefore constitutes the
{\em hypothesis} for this dissertation and will be
further elaborated and validated.

Our initial observation is that software is extremely
complex, i.e., that ``software systems are perhaps the
most intricate of man's handiworks''~\cite{Brooks:Anniversary}.
Software complexity is not only ubiquitous, it is also
{\em intrinsic}:
``The complexity of software is an essential property,
not an accidental one''~\cite{brooks87}.  ``By essential
we mean that we may master this complexity, but we can
never make it go away''~\cite{Booch:OOAD}.
Complexity hinders software understanding, yet understanding
must be afforded if development is to be successful.

To manage complexity, software development concerns
are often separated in time and space.  Thus, software
is typically developed in distinct phases by different
teams of developers.  Large-scale software is often
long-lived and delivered in multiple version, thereby
comprising a family of related programs~\cite{Parnas:Extension}.

The product of software development is typically a
large and complex collection of software elements of
diverse types.  Software artifact collections often
contain not only millions of source code lines,
but also versioned releases, volumes of user 
manuals and documentation, requirements and design
specifications, test cases and test results, and so forth.
Moreover, these software objects are typically
integrated and correlated within themselves and
with each other in surprisingly intricate and subtle
ways~\cite{Steinberg:Yosemite}.  This poses
substantial impediments to software understanding,
particularly with respect to traceability,
visualization, and navigation issues, thereby
leading to the following research question.

{\bf Research Question}:
\begin{quote}
Given the intricacy and complexity of software
artifact relationships, what can be done to
improve traceability, visualization, and
navigation of large artifact collections?
\end{quote}

As a first step, it is proposed that software systems
be viewed as large hypertexts, since hypertext offers
a coherent and consistent metaphor for viewing both
inter-- as well as intra--artifact relationships
(cf.~\cite{Ziv:ResearchIssues,Steinberg:Yosemite}).
Here, it suffices to define hypertext informally
as providing node-and-link views and navigation
of software artifact collections.  Hypertext is
defined more formally and completely in
Chapter~\ref{background}.  We therefore contend
that a hypertext metaphor of software artifacts
and relations should improve developer's ability
to trace or track related elements in large and
complex software spaces.

Despite the above, hypertext is no ``silver bullet''
for relieving software artifact complexities and
uncertainties.  Specifically, hypertext introduces
several concerns regarding its efficacy for
software engineering~\cite{Ziv:ResearchIssues}.
Here, we are particularly concerned with the
well-known navigation problem of user confusion
and disorientation in large hypertext spaces as
well as the fact that hypertext systems generally
are not designed with software engineering in mind.
This means that hypertext does not necessarily
address the unique needs of software developers
and users nor does it model or make use of the
unique characteristics of software systems.
Additional means should, therefore, be provided to further
facilitate understanding of software artifact collections.
The following statements concern this facilitation.

We claim that software engineering is fraught
with uncertainties.  Software uncertainties
contribute significantly to the overall complexity 
and unpredictability of software development.
Like complexity, uncertainty is inherent to the
engineering of software systems.  This observation
is summarized succinctly in~\cite{Ziv:MUSE} as the 
{\em Maxim of Uncertainty in Software Engineering}:
\begin{quote}
{\em
Uncertainty is inherent and inevitable in software processes
and products.
}
\end{quote}

Examples of software uncertainties abound -- many are provided
in this dissertation.  Starting in Chapter Four, we focus on
uncertainties associated with developers' confidence levels
in software artifacts, including, among others, requirements
specifications, design elements, code modules, and testing
information.  These confidence levels fluctuate frequently
during development.  Their modeling should, therefore,
include a scheme for confidence revision and updating.

We now arrive at the high-level statement of the hypothesis
for this dissertation.

{\bf High-level Research Hypothesis}:
\begin{quote}
Explicitly modeling uncertainty improves human judgment and
decision making during software development. 
\end{quote}

This hypothesis is too vague to be investigated and
validated directly.  Instead, we require a more specific
formulation, including a specific technique for software
uncertainty modeling.
To model software uncertainties, we turn to artificial
intelligence research in modeling and management of
uncertainty.  We select a specific uncertainty modeling
technique called Bayesian belief networks~\cite{Pearl88}.
Detailed reasons for this choice are provided in Chapter Four.

Here, it suffices to say that Bayesian networks
offer a clear conceptual model of causality among related
elements and include algorithms for belief revision and
updating.  This leads to our specific hypothesis below.
The hypothesis speaks in terms of software artifacts,
which include, among others, requirements elements,
design nodes, code modules, and test information,
relationships among those artifacts, established as
part of the development process, and confidence
levels, which reflect developer confidences in
certain qualities and properties of software artifacts.

{\bf Specific Research Hypothesis}:
\begin{quote}
Bayesian-network models of software artifact uncertainties improves
understanding of associated confidence levels, compared to simply
following linked information, ultimately leading to better human
decision making.
\end{quote}

In the long run, we believe that proper adoption and use of
uncertainty modeling techniques will improve human judgment
and decision making during software development.  For the
purposes of this dissertation, we focused on the specific
hypothesis above and took steps toward its validation,
as follows:
\begin{enumerate}
\item
An existing software system was selected as case study.
The system of choice, called CEquencer, is currently
under development at Beckman Instruments in Fullerton,
CA.  CEquencer is designed to control and communicate
with various hardware devices that in turn are used by 
biologists, chemists, and other scientists to separate
laboratory specimens into molecular constituents in
order to determine their DNA sequences.
CEquencer software artifacts are described in Chapter Five.
\item
CEquencer requirements were investigated throughly,
and a hypertext model as well as a Bayesian-network
model of those requirements were constructed.
\item
A key subsystem of CEquencer, called FSM Run, was
investigated thoroughly, resulting in a hypertext
model of software artifacts, including all FSM Run
code modules as well as related requirements.
\item
Software uncertainty information was collected for FSM Run
artifacts.  The information was elicited from Beckman
experts and developers and organized in a Bayesian network.
\item
Our main hypothesis was evaluated against FSM Run code
modules and related requirements, as identified in (2)
above.  In a questionnaire session with the Beckman team,
developers were asked to trace artifacts and associated
confidence levels with and without the Bayesian-network
representation of confidence values.
\item
Results were gathered for the case study and analyzed
for statistical significance.  Results of the
questionnaire session are summarized in Chapter Six.
\end{enumerate}

\section{Dissertation Contributions}

We consider the key contributions of this dissertation to be:
\begin{enumerate}
\item
Defining succinctly the Maxim of Uncertainty in Software
Engineering, identifying the abundance of uncertainties
and their relevance to software process decisions and
project risk management.  Also, as corollary, we suggest
strongly that the software life cycle include search,
identification, modeling and management of software 
uncertainties.
\item
Defining an approach to explicitly model uncertainty
in relevant software situations.  This approach is
anchored in a Bayesian interpretation of relationships
and dependencies among software artifacts.  It is
suggested and demonstrated that Bayesian belief
networks, originally described by
Pearl~\cite{Pearl88}, may be used for this purpose.
\item
Implementing a prototype Java applet that allows software
systems to be defined as graphs of related artifacts
as well as be annotated with uncertainty information.
These software belief networks can then be subject to
Bayesian updating.
\item
Conducting a case study to substantiate the maxim above and 
to evaluate the applicability of Bayesian belief networks 
to a real software project.  To this end, we selected as
case study the CEquencer system described earlier.
Though confined, our case study indicates that explicit
modeling of software uncertainties improves developers'
ability to identify and track changes to levels of
confidence in software artifacts and relations.
\end{enumerate}

\section{Dissertation Organization}

The rest of this dissertation is organized as follows:
In Chapter Two we present background information in the
form of a brief historical perspective of thirty years
of related software engineering research.
Specifically, Chapter Two covers areas of software
visualization, hypertext and hypermedia, object-oriented
methods, and AI inference diagrams.  Chapter Three
discusses software uncertainty in detail, including 
sources of uncertainty and uncertainty in development
phases.
In Chapter Four we present an uncertainty modeling
technique called Bayesian Belief Networks and justify
its applicability to software engineering situations.
Several examples of Bayesian networks are included.
Chapter Five describes the architecture and components
of the CEquencer system, selected as case study for
this dissertation.  In Chapter Six we present results
and impressions from our study of uncertainty modeling
at Beckman.  Chapter Seven describes three specific
applications of Bayesian networks in practice as well
as related work in uncertainty modeling.  Finally,
Chapter Eight presents conclusions and future work.

In conclusion of the introduction, we wish to point out
that uncertainty abounds not only in software development
but also in most engineering and scientific pursuits
as well as many everyday situations (For uncertainty in
everyday situations see, for example, Stefik~\cite{Stefik:KBSE},
pp. 460.).  Detailed exploration of uncertainty in general
is, therefore, beyond the scope of this dissertation.
Nevertheless, we hope that the maxims, techniques, tools,
and results described here will help identify research
opportunities as well as provide a solid foundation for
future work in software uncertainty modeling.  We
encourage readers to consider occurrences and consequences
of uncertainty in their own domains of interest and expertise.
%
Our future plans include additional modeling of software
uncertainties at Beckman as well as exploring other domains
where uncertainty modeling may be useful.  We are particularly
interested in ``Bayesian Internets,'' i.e., the modeling of
information retrieval uncertainties on the Internet and WWW.
