(Last modified Thu Jun 05 22:59 2008)

Some definitions of software quality

IEEE Standard Glossary of Software Engineering Terminology (Galin p24):  "The degree to which a system, component, or process meets specified requirements."  (Determining this is verification.)  "The degree to which a system, component, or process meets customer or user needs or expectations."  (Determining this is validation.) 

Pressman Software Engineering -- A Practitioner's Approach:  "Software quality is defined as:  Conformance to explicitly stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software."

[original figure]

Boehm, Brown, and Lipow 1976

Why ilities are important

‘Suppose you receive a software product which is delivered on time, within budget, and which correctly and efficiently performs all its specified functions.  Does it follow that you will be happy with it?  For several reasons, the answer may be "no."  Here are some of the common problems you may find: 

1. ‘The software product may be hard to understand and difficult to modify.  This leads to excessive costs in software maintenance, and these costs are not trivial.  For example, a recent paper by Elshoff (1) indicates that 75 percent of General Motors' software effort is spent in software maintenance, and that GM is fairly typical of large industry software activities.
2. ‘The software product may be difficult to use, or easy to misuse.  A recent GAO report (2) identified over $I0,000,000 in unnecessary Government costs due to ADP problems; many of them were because the software was so easy to misuse. 
3. ‘The software product may be unnecessarily machine-dependent, or hard to integrate with other programs.  This problem is difficult enough now, but as machine types continue to proliferate, it will get worse and worse.’

[Boehm+Brown+Lipow1976-qesq p.592]

Boehm, Brown, and Lipow's definitions of 23 ilities

(These are unusually full definitions; most authors are so brief that a reader must define the qualities for him/herself.) 

ACCESSIBILITY

Code possesses the characteristic accessibility to the extent that it facilitates selective use of its parts.  (Examples:  variable dimensioned arrays, or not using absolute constants.)  Accessibility is necessary for efficiency, testability, and human engineering. 

ACCOUNTABILITY

Code possesses the characteristic accountability to the extent that its usage can be measured.  This means that critical segments of code can be instrumented with probes to measure timing, whether specified branches are exercised, etc.  Code used for probes is preferably invoked by conditional assembly techniques to eliminate the additional instruction words or added execution times when the measurements are not needed. 

ACCURACY

Code possesses the characteristic accuracy to the extent that its outputs are sufficiently precise to satisfy their intended use.  Necessary for reliability. 

AUGMENTABILITY

Code possesses the characteristic augmentability to the extent that it can easily accommodate expansion in component computational functions or data storage requirements.  This is a necessary characteristic for modifiability. 

COMMUNICATIVENESS

Code possesses the characteristic communicativeness to the extent that it facilitates the specification of inputs and provides outputs whose form and content are easy to assimilate and useful.  Communicativeness is necessary for testability and human engineering. 

COMPLETENESS

Code possesses the characteristic completeness to the extent that all its parts are present and each part is fully developed.  This implies that external references are available and required functions are coded and present as designed, etc. 

CONCISENESS

Code possesses the characteristic conciseness to the extent that excessive information is not present.  This implies that programs are not excessively fragmented into modules, overlays, functions and subroutines, nor that the same sequence of code is repeated in numerous places, rather than defining a subroutine or macro; etc. 

CONSISTENCY

Code possesses the characteristic internal consistency to the extent that it contains uniform notation, terminology and symbology within itself, and external consistency to the extent that the content is traceable to the requirements.  Internal consistency implies that coding standards are homogeneously adhered to; e.g., comments should not be unnecessarily extensive or wordy at one place, and insufficiently informative at another, that number of arguments in subroutine calls match with subroutine header, etc.  External consistency implies that variable names and definitions, including physical units, are consistent with a Glossary; or, there is a one-one relationship between functional flow chart entities and coded routines or modules, etc. 

DEVICE-INDEPENDENCE

Code possesses the characteristic device-independence to the extent it can be executed on computer hardware configurations other than its current one.  Clearly this characteristic is a necessary condition for portability. 

EFFICIENCY

Code possesses the characteristic efficiency to the extent that it fulfills its purpose without waste of resources.  This implies that choices of source code constructions are made in order to produce the minimum number of words of object code, or that where alternate algorithms are available, those taking the least time are chosen; or that information-packing density in core is high, etc.  Of course, many of the ways of coding efficiently are not necessarily efficient in the sense of being cost-effective, since portability, maintainability, etc., may be degraded as a result. 

HUMAN ENGINEERING

Code possesses the characteristic human engineering to the extent that it fulfills its purpose without wasting the users' time and energy, or degrading their morale.  This characteristic implies accessibility, robustness, and communicativeness. 

LEGIBILITY

Code possesses the characteristic legibility to the extent that its function is easily discerned by reading the code.  (Example:  complex expressions have mnemonic variable names and parentheses even if unnecessary.)  Legibility is necessary for understandability. 

MAINTAINABILITY

Code possesses the characteristic maintainability to the extent that it facilitates updating to satisfy new requirements or to correct deficiencies.  This implies that the code is understandable, testable and modifiable; e.g., comments are used to locate subroutine calls and entry points, visual search for locations of branching statements and their targets is facilitated by special formats, or the program is designed to fit into available resources with plenty of margins to avoid major redesign, etc. 

MODIFIABILITY

Code possesses the characteristic modifiability to the extent that it facilitates the incorporation of changes, once the nature of the desired change has been determined.  Note the higher level of abstractness of this characteristic as compared with augmentability. 

PORTABILITY

Code possesses the characteristic portability to the extent that it can be operated easily and well on computer configurations other than its current one.  This implies that special language features, not easily available at other facilities, are not used; or that standard library functions and subroutines are selected for universal applicability, etc. 

RELIABILITY

Code possesses the characteristic reliability to the extent that it can be expected to perform its intended functions satisfactorily.  This implies that the program will compile, load, and execute, producing answers of the requisite accuracy; and that the program will continue to operate correctly, except for a tolerably small number of instances, while in operational use.  It also implies that it is complete and externally consistent, etc. 

ROBUSTNESS

Code possesses the characteristic robustness to the extent that it can continue to perform despite some violation of the assumptions in its specification.  This implies, for example, that the program will properly handle inputs out of range, or in different format or type than defined, without degrading its performance of functions not dependent on the non-standard inputs. 

SELF-CONTAINEDNESS

Code possesses the characteristic self-containedness to the extent that it performs all its explicit and implicit functions within itself.  Examples of implicit functions are initialization, input checking, diagnostics, etc. 

SELF-DESCRIPTIVENESS

Code possesses the characteristic self-descriptiveness to the extent that it contains enough information for a reader to determine or verify its objectives, assumptions, constraints, inputs, outputs, components, and revision status.  Commentary and traceability of previous changes by transforming previous versions of code into non-executable but present (or available by macro calls) code are some of the ways of providing this characteristic.  Self-descriptiveness is necessary for both testability and understandability.

STRUCTUREDNESS

Code possesses the characteristic structuredness to the extent that it possesses a definite pattern of organization of its interdependent parts.  This implies that evolution of the program design has proceeded in an orderly and systematic manner, and that standard control structures have been followed in coding the program, etc. 

TESTABILITY

Code possesses the characteristic testability to the extent that it facilitates the establishment of verification criteria and supports evaluation of its performance.  This implies that requirements are matched to specific modules, or diagnostic capabilities are provided, etc. 

UNDERSTANDABILITY

Code possesses the characteristic understandability to the extent that its purpose is clear to the inspector.  This implies that variable names or symbols are used consistently, modules of code are self-descriptive, and the control structure is simple or in accordance with a prescribed standard, etc. 

USABILITY (AS-lS UTILITY)

Code possesses the characteristic usability to the extent that it is reliable, efficient and human-engineered.  This implies that the function performed by the program is useful elsewhere, is robust against human errors (e.g., accepts either integer or real representations for type real variables), or does not require excessive core memory, etc. 

[Boehm+Brown+Lipow1976-qesq pp. 603-605]

Cavano and McCall's definitions of the 11 quality factors 1978

Cavano and McCall give a brief definition of each of the 12 factors, plus (most usefully) give a question for each factor, in the figure. 

CORRECTNESS

Extent to which a program satisfies its specifications and fulfills the user's mission objectives.

RELIABILITY

Extent to which a program can be expected to perform its intended function with required precision.

EFFICIENCY

The amount of computing resources and code required by a program to perform a function.

INTEGRITY

Extent to which access to software or data by unauthorized persons can be controlled.

USABILITY

Effort required to learn, operate, prepare input, and interpret output of a program.

MAINTAINABILITY

Effort required to locate and fix an error in an operational program.

TESTABILITY

Effort required to test a program to ensure it performs its intended function.

FLEXIBILITY

Effort required to modify an operational program.

PORTABILITY

Effort required to transfer a program from one hardware configuration and/or software system environment to another.

REUSABILITY

Extent to which a program can be used in other applications - related to the packaging and scope of the functions that programs perform.

INTEROPERABILITY

Effort required to couple one system with another.

[Cavano+McCall1978-fmsq]

Blundell, Hines, and Stach 1997

Blundell et al. present this table of 40 ilities in their paper on software quality metrics.  Some of their ility descriptions are essentially lists of quantities that can be measured. 

Distinguishing two ilities

How could we decide two qualities are distinct?

• If different stakeholders benefit from each of the two qualities
• If the effect of the two qualities are seen in different phases of the lifecycle
• If the qualities are manifested in two different ways
• If there are systems or situations for which only one of the two qualities might be desirable

References

Blundell+Hines+Stach1997-msdq  James Kenneth Blundell, Mary Lou Hines, and Jerrold Stach.  The measurement of software design quality.  Ann. Softw. Eng., 4 235-255 1997.  doi

Boehm+Brown+Lipow1976-qesq  B. W. Boehm, J. R. Brown, and M. Lipow.  Quantitative evaluation of software quality.  In ICSE '76:  Proceedings of the 2nd International Conference on Software Engineering, pp. 592-605.  IEEE Computer Society Press, 1976.  url

Cavano+McCall1978-fmsq  Joseph P. Cavano and James A. McCall.  A framework for the measurement of software quality.  In Proceedings of the Software Quality Assurance Workshop on Functional and Performance Issues, pp. 133-139.  1978.  doi

Thomas A. Alspaugh
Assistant Professor, Informatics Dept.
School of Information and Computer Sciences