Analyzing Test Data Selection Criteria
"An Analysis of Test Data Selection Criteria using the RELAY Model of
Fault Detection,"
by D. Richardson and M.C. Thompson,
in IEEE Transactions on Software Engineering, June 1993
Abstract
RELAY is a model of faults and failures that defines failure
conditions, which describe test data for which execution will
guarantee that a fault originates erroneous behavior that also
transfers through computations and information flow until a
failure is revealed. This model of fault detection provides a
framework within which other testing criteria's capabilities can be
evaluated. In this paper, we analyze three test data selection
criteria that attempt to detect faults in six fault classes. This
analysis shows that none of these criteria is capable of guaranteeing
detection for these fault classes and points out two major weaknesses
of these criteria. The first weakness is that the criteria do not
consider the potential unsatisfiability of their rules; each criterion
includes rules that are sufficient to cause potential failures for
some fault classes, yet when such rules are unsatisfiable, many faults
may remain undetected. Their second weakness is failure to integrate
their proposed rules; although a criterion may cause a subexpression
to take on an erroneous value, there is no effort made to guarantee
that the intermediate values cause observable, erroneous behavior.
This paper shows how the RELAY model overcomes these weaknesses.
from Debra J. Richardson
djr@ics.uci.edu
Department of Information and Computer Science,
University of California, Irvine CA 92717-3425