\chapter{Conclusions and Future Work}
\label{discussion}

\section{Summary of Contributions}

This dissertation offers the following contributions:
\begin{itemize}
\item
{\em Defining the Maxim of Uncertainty in Software
Engineering}.  This maxim recognizes the abundance
of software uncertainties and their relations and
impacts on process decisions and risk management.
Also, as corollary, it is suggested that software
development include the search,
identification, modeling and management of software
uncertainties.
\item
{\em Defining an approach to explicitly model uncertainty
using software belief networks}.  This approach
is anchored in a Bayesian interpretation of relationships
and dependencies among software artifacts.  It includes
notions of human judgment, confidence, belief, and evidence,
often ignored by software modeling techniques.
\item
{\em Implementing the SoftBayes Java applet}.
This applet allows software systems to be defined
as graphs of related artifacts and to be annotated
with uncertainty information.
Once defined, software belief networks are subject to
Bayesian updating, using a core engine such as provided
by {\em JavaBayes}.
\item
{\em Conducting a case study to substantiate the maxim above and
to evaluate the applicability of Bayesian belief networks
to a real software project}.  Results of the CEquencer study
indicate that explicit
modeling of software uncertainties improves developers'
ability to identify and track changes to levels of
confidence in software artifacts and relations.
\item
{\em Finally, several confidence factors were identified
as influencing expert's beliefs in software requirements}.
Recording those confidence factors is deemed beneficial
to future similar projects where requirements uncertainty
is captured explicitly.
\end{itemize}

Several impediments
and limitations of our approach were observed, including
the initial cost of obtaining prior belief values, the
need to ensure that software belief networks retain
causality and variable independence, and the assumption
that software developers, domain experts, and related
project information are available and accessible.

Our experience in using Bayesian networks for software
uncertainty modeling, specifically for the CEquencer
system, indicates that:
\begin{itemize}
\item
The conceptual view of both software systems and Bayesian networks
as interrelated ``webs'' of nodes and links seems to offer a
convenient metaphor that also maps well into subsequent design
and implementation.  Specifically, early depictions of CEquencer
artifact webs were developed as Java applets and placed on the
World Wide Web (see~\cite{Ziv:applets});  these applets were
then viewed and reviewed by Beckman developers for accuracy
and relevancy.
\item
The CEquencer system, like most other software, proved to be
fraught with uncertainties, thereby confirming our suspicions
for at least one real-life system.  Specifically, CEquencer
software embodies many, often subtle, problem domain
uncertainties, including uncertainties stemming from
laws of physics and chemistry in the software's operational
environment as well as from vaguely defined boundaries
between software versus hardware components.
\item
Our notion of {\em software uncertainties} was
well received by Beckman developers, offering convenient
means in which to describe their confidences and beliefs
regarding CEquencer software.  The CEquencer requirements
analyst has recently incorporated confidence levels into
her requirements capture process.
\item
The questionnaire session at Beckman revealed that,
for the given set of questions, a significant
difference between the means of the subjects' scores was
identified.  This suggests that subjects' ability to
answer the questions increased with the addition of
confidence level information.  This confirms a
statistically significant relationship between the
availability of confidence level information and
developers' ability to trace and track software
artifacts and related confidences.
\end{itemize}

\section{Future Work}

%% We believe that much more stands to be gained by explicit
%% modeling of uncertainty in software engineering.  In
%% remaining paragraphs, we discuss additional uses and future
%% research directions for uncertainty modeling.

\subsection{Comprehensive Case Study}

As discussed in chapter~\ref{results}, the case study at Beckman
was confined to a single group of developers, a single Bayesian
model of confidence-level uncertainty, and a single scenario of
change with associated questions.  We are very interested in
expanding the scope of our field studies, both within Beckman
as well as with other software development organizations.

A more comprehensive study than the one reported in this
dissertation would include one or more of the
following enhancements: 
\begin{itemize}
\item
{\em Measure the effects of using the Bayesian models over time,
especially its impacts on the quality of human judgment and
decision making}.
\item
{\em Construct additional Bayesian models for additional artifact
networks or to model other situations or properties}.
We are particularly interested in constructing Bayesian models
at different levels of granularity, comparing the efficacy of
resulting models, and, eventually, combining Bayesian models
with different granularities.
\item
{\em Afford automated support for definition and revision of 
software belief networks}.  Of special interest are UI design
issues in constructing belief-network software as well as
integration
of Bayesian capabilities with existing tools, e.g., REBUS,
DOORS, and RequisitePro.
\item
{\em Finally, this dissertation focused on incorporation of
subjective information into software models}.  We are
interested in combining subjective and objective data
in future models.  Specifically, we wish to include
project data such as code quality and developer
productivity, failure rate and severity, etc., in our
models as factors that influence confidence and,
ultimately, human judgment and decision making.
\end{itemize}

\subsection{Monitoring the testing process}
We intend to study the relationship between the
Bayesian approach and software testing.  This
relationship is important not only since testing
should have significant implications for belief
and confidence levels, but also since a Bayesian
model should have ramifications for human judgment
and decision making during testing.  For example,
one important question in software testing is
``How much testing is enough?''.  This question
may be addressed by explicit modeling of uncertainty,
if sufficient testing is defined in terms of
levels of confidence in select system entities,
for example, its code modules.  As testing progresses,
confidence levels increase as long as test execution
is successful.  Testing is guided and monitored by
%The amount by which
%confidence increases depends on multiple factors, including,
%for example, one's confidence in the testing strategy being used.
continuous update and comparison of confidence levels against
predefined thresholds.  Testers are notified and may take appropriate
action whenever thresholds are exceeded.  This approach may be
especially useful in safety-critical systems, where confidence
requirements and constraints
are often specified numerically.
With regard to CEquencer, we intend to enhance our Bayesian
model to include testing information - as it becomes available
at Beckman - and monitor the effects of the Bayesian approach
on testing decisions.

\subsection{Other software qualities}
In this dissertation, we focused on implementability
and later validity as interesting software qualities
for which causal relationships can be established
and belief values can be determined.  We believe,
however, that uncertainty should also be modeled
for additional software qualities, including,
among others, correctness, reliability, efficiency,
safety, and maintainability.  Of particular
relevance is the work carried out at CSR, discussed
in chapter~\ref{relwork}, where uncertainty is
modeled for software testability and dependability
assessment tasks.
As mentioned earlier, qualities associated with entities must
be consistent with causal relationships such that the resulting
network is meaningful.

\subsection{Other uncertainty modeling techniques}
In this dissertation, we have used Bayesian networks
to model uncertainty in software development.
Alternative approaches to uncertainty modeling and
logical inference exist, however, that must be weighed
and compared against Bayesian methods.  Those include
Certainty--Factor approaches, Dempster--Shafer approaches,
fuzzy logic, and default, monotonic, and nonmonotonic
logic~\cite{Stefik:KBSE}.  Specifically, relative merits
and pitfalls of these techniques should be evaluated
against the Bayesian
approach in the context of software engineering situations.

\subsection{Modeling uncertainty in software processes}
In this dissertation, we focused on software product
uncertainties, such as modeled for CEquencer software,
though process uncertainties were treated briefly
(e.g., path selection criteria in Table~\ref{Assignment}).
Despite the emphasis on artifact uncertainties, we firmly
believe that process uncertainties should also be modeled.
We contend
that software-process modeling formalisms must be augmented
to include uncertainty values; that an environment for
supporting definition and execution of process models should
include capabilities for representation and interpretation of
belief values and should allow for Bayesian updating of those
values; and that Bayesian updating procedures must be carried
out during process execution, such that belief values and
confidence levels are continuously updated as new evidence arrives.

The provision and update of belief values may be greatly enhanced
in software process frameworks that include process measurement
capabilities.  Such capabilities constitute a rich source of
information regarding the current state of various elements and
support the collection of statistical and empirical data that
may significantly improve the accuracy of prior belief value estimation.

We expect that by modeling software process uncertainties,
one may achieve a more realistic representation of the process,
enable automated belief revision by means of Bayesian updating,
and support prediction and guidance of future development activities.

\section{Final Thoughts}

This dissertation offers a path that, if followed, helps address
software development problems for which uncertainty plays an
important role, specifically with respect to human judgment
and decision making.  We strongly believe that software
uncertainty modeling - using Bayesian belief networks -
will help managers to better plan, schedule, estimate, and evaluate
software projects.  It will also help developers to better trace
and navigate interrelated artifacts in large software spaces,
understand and discover system qualities and properties, and,
ultimately, achieve better judgment and make better decisions.

