Workshop Description

ROSATEA is a cooperative, international workshop specifically aimed at investigating the Role Of Software Architecture in Testing E (and) Analysis of complex systems. The study of software architecture has lately become an autonomous discipline [SG96, ISAW96, Wol97] and the field of testing and analysis has matured substantially in recent years [ISSTA96]. Thus, the interdisciplinary investigation of how these two disciplines can best leverage one another's technological advances is timely [RW96, ER96, GK96]. This workshop will investigate the many potential roles of software architecture in testing and analyzing complex systems, highlighting the numerous advantages of exploiting formal software architecture specifications in this pursuit, as well as establish a research agenda for this cross-disciplinary investigation.

The primary participants at this workshop will be U.S. and Italian researchers who have expertise and focused research programs in either software architecture or testing and analysis as well as an interest in the interaction between the two disciplines. Each country has prominent researchers in both disciplines, yet the researchers between countries have even more complimentary expertise due to different foci. The workshop is sponsored by the U.S. National Science Foundation (NSF) and the Italian National Research Council (CNR). The U.S. workshop chair and organizer is Professor Debra J. Richardson of the University of California at Irvine. The Italian organizers are Professor Paola Inverardi of the Università dell'Aquila (workshop chair) and Antonia Bertolino of the Instituto di Elaborazione della Informazione at CNR (local arrangements).

Software engineers have long recognized the importance of supplying information about how their software is structured in components and how those components interact. To be most useful, software architectures must be described at a high level of abstraction that is comprehensible to a variety of users who have different educational backgrounds, perspectives, and goals [PW92, GS93]. Most efforts in the study of Software Architecture have focused on the description, design and construction of the system structure [AG97, IW95, LeM96, LV95, MS96, MT97]. As the complexity of software systems has increased, however, the importance of the role of software architecture in the testing and analysis of the system has become evident as well [LKA+95, BI96, ER96, ER97, MR97, RW96].

Testing and analysis of complex systems can be facilitated by software architecture description, which supports reasoning about how system requirements are satisfied in terms of assigning functionality to components and in terms of the interaction of those components via their connectors. The emerging formal notations for software architecture specification [GMW97, IW95, LV95, MK96, MOEK95, Ves96] provide a sound basis upon which to develop architecture-based testing and analysis techniques and tools. Furthermore, the technological advances in testing and analysis [CCM96, CGPV97, Jac96, RH96, Ros97, PY97] have provided approaches that could be applied at various levels of abstraction, including the architectural level. Yet cross-disciplinary research has received only limited attention thus far. Software architectures exhibit both static (structural) and dynamic (behavioral) features, and software architecture specifications can indeed provide formal and explicit descriptions of both the static and the dynamic properties of a system. Thus, it has been proposed to employ software architecture specifications to devise and control both static analysis techniques [CBKA95, NCO97, SRW97] and dynamic analysis techniques [BI96, RW96, Ore96] so that their focus is on uncovering software architectural defects, where the interaction or communication among software components is incorrect.

Testing and analyzing at the architectural level has several benefits. For one, it enables focusing on architectural defects rather than relying on other test strategies to detect these defects. Additionally, it facilitates detecting such defects early in the software life-cycle, rather than after implementation and during system integration, as is typical not to mention overly costly. Furthermore, since an architecture is often reused to develop multiple systems, the cost of any architecture level testing effort is amortized across the family of systems, thus leveraging testing costs, which are extremely high relative to the rest of development costs. The cross-disciplinary investigation of software architecture with testing and analysis is, therefore, a research topic worthy of a focused workshop.

The objectives of the workshop are to ascertain the applicability of formalized software architecture specification as a basis for analysis and testing, to judge the most appropriate formalisms for software architecture specifications for this purpose, to judge the best uses for software architecture information in analysis and testing, to establish a research agenda for cross-disciplinary investigation, and to characterize a set of challenge problems for researchers in the area. Example topics relevant to this workshop include:

formal software architecture description techniques that facilitate analysis and testing;
static analysis techniques for identifying faulty properties of software architecture specifications;
flow analysis techniques for detecting inconsistent component interactions allowed by a software architecture specification;
measures of system testability based on software architecture and their use in architectural choice and test planning;
architecture-based test criteria focused on detecting architectural defects;
exploitation of architectural styles and domain-specific architectures for testing and analyzing families of systems sharing an architecture;
conformance testing to assess consistency between an architecture specification and its implementations;
dynamic and static slicing techniques for localizing architectural defects, evaluating change impact, and re-architecting;
effective regression testing and analysis of evolving systems based on architecture evolution;
analysis techniques for reasoning about interaction between architectural components and the external environment;
automated software testing and analysis processes based on software architecture descriptions.

The workshop will be structured around a selected set of topics and issues with respect to the role of software architecture in testing and analysis. This set will be determined by the organizers on the basis of position papers submitted by invited participants. The workshop will consist of several focused sessions, each leading off with an introductory talk by one selected participant, which will set the stage for scheduled and impromptu presentations by other participants and follow-up discussion. All attendees will participate in each session. The agenda below is only a starting point upon which feedback and suggestions will be raised.

A final report will be written by the workshop co-organizers and reviewed by selected workshop participants. This report will include: 1) summary of workshop results; 2) position papers; 3) research agenda; 4) community challenge problems; 5) plans for future cooperative activities. The final report will be included in the workshop succeedings, which will be sent to all participants, sponsors, and other interested parties. Plans are to submit the final report to a widely-disseminated publication.

Bibliography

[AG97] R. Allen and D. Garlan. A formal basis for architectural connection. ACM Transactions on Software Engineering and Methodology, 6(3):213-249, July 1997.

[BI96] A. Bertolino and P. Inverardi. Architecture-based software testing. In [ISAW96], October 1996.

[CBKA95] P. Clements, L. Bass, R. Kazman, and G. Abowd. Predicting software quality by architecture-level evaluation," in Proceedings of the Fifth International Conference on Software Quality. Austin, Texas, October 1995.

[CCM96] P. Ciancarini, S. Cimato, C. Mascolo. Engineering formal requirements: analysis and testing. Proceedings of the 1996 Conference on Software Engineering and Knowledge Engineering (SEKE'96), 1996.

[CGPV97] C. Cugola, C. Ghezzi, G.P.Picco, G. Vigna, Analyzing Mobile Code Languages, in Mobile Object Systems towards the Programmable Internet, Jar. Vitek and C. Ts chudin, eds., LNCS, Springer-Verlag, April 1997

[ER96] N. S. Eickelmann and D. J. Richardson. What makes one software architecture more testable than another? In [ISAW96], October 1996.

[ER97] N. S. Eickelmann and D. J. Richardson. An evaluation of measurement theoretic models of software testability. UCI TR-97, May 1997.

[GS93] D. Garlan and M. Shaw. An Introduction to Software Architecture. In V. Ambriola and G. Tortora, editors, Advances in Software Engineering and Knowledge Engineering, volume 1. World Scientific, New Jersey, 1993.

[GMW97] D. Garlan, R. Monroe, and D. Wile. ACME: An Architecture Description Interchange Language. Proceedings of CASCON'97, IBM Center for Advanced Studies. November 1997.

[GK96] W. Griswold and P. Kruchten. Tools and Methods group report. In Succeedings of the Second International Software Architecture Workshop. ACM SIGSOFT Software Engineering Notes, 22( 1): 42-56, January 1997.

[IW95] P. Inverardi and A. Wolf. Formal Specification and Analysis of Software Architectures using the Chemical Abstract Machine Model. IEEE Transactions on Software Engineering, 21(4):373-386, Apr. 1995.

[ISAW96] Proceedings of the Second International Software Architecture Workshop (ISAW-2). In L. Vidal, A. Finkelstein, G. Spanoudakis, and A.L.Wolf (eds) Joint Proceedings of the SIGSOFT '96 Workshops, Part1, San Francisco, October 1996, ACM Press.

[ISSTA96] Proceedings of the 1996 International Symposium on Software Testing and Analysis (ISSTA '96). ACM SIGSOFT, Jan. 1996.

[Jac96] D. Jackson. Nitpick: A checkable specication language. In Proceedings of the Workshop on Formal Methods in Software Practice, San Diego, CA, January 1996.

[LeM96] D. Le Metayer, Software Architecture styles as graph grammars. InProceedings of the 4th ACM SIGSOFT Symposium on Foundations of Software Engineering, ACM Press, 1996.

[LKA+95] D. Luckham, J. Kenney, L. Augustin, J. Vera, D. Bryan, and W. Mann. Specification and Analysis of System Architecture Using Rapide. IEEE Transactions on Software Engineering, 21(4):336-355, April 1995.

[LV95] D. Luckham and J. Vera. An Event-based Architecture Definition Language. IEEE Transactions on Software Engineering, 21(9):717-734, September 1995.

[MDEK95] J. Magee, N. Dulay, S. Eisenbach, and J. Kramer. Specifying Distributed Software Architectures. In Proceedings of the Fifth European Software Engineering Conference (ESEC'95), Barcelona, September 1995.

[MK96] J. Magee and J. Kramer. Dynamic Structure in Software Architectures. In Proceedings of ACM SIGSOFT'96: Fourth Symposium on the Foundations of Software Engineering (FSE4), 3-14, San Francisco, CA, October 1996.

[MR97] N. Medvidovic and D. Rosenblum. Domains of Concern in Software Architectures and Architecture Description Languages. In Proceedings of the 1997 USENIX Conference on Domain-Specific Languages, Santa Barbara, California, October 1997.

[MS96] C. Montangero and L. Semini. Refining by architectural style or architecting by refinements. In [ISAW96], 76-79.

[MT97] N. Medvidovic and R. N. Taylor. A Framework for Classifying and Comparing Architecture Description Languages. In Proceedings of the Sixth European Software Engineering Conference together with Fifth ACM SIGSOFT Symposium on the Foundations of Software Engineering, 60-76, Zurich, Switzerland, September 1997.

[NCO97] G. Naomavich, G.S. Avrunin, L.A. Clarke, and L.J. Osterweil. Applying Static Analysis to Software Architectures. In Proceedings of the Sixth European Software Engineering Conference together with Fifth ACM SIGSOFT Symposium on the Foundations of Software Engineering, 77-93, Zurich, Switzerland, September 1997.

[Ore96] P. Oreizy. Issues in the Runtime Modification of Software Architectures. Technical Report, UCI-ICS-96-35, University of California, Irvine, August 1996.

[PW92] D. Perry and A. Wolf. Foundations for the Study of Software Architecture. SIGSOFT Software Engineering Notes, 17(4):40-52, October 1992.

[PY97] M. Pezze and M. Young. Constructing multi-formalism state-space analysis tools: using rules to specify dynamic semantics of models. In Proceedings of the 19th International Conference on Software Engineering, 239-249, Boston, MA, May 1997.

[RW96] D. J. Richardson and A. L. Wolf. Software testing at the architectural level. In [ISAW96], October 1996.

[Ros97] D. Rosenblum. Adequate Testing of Component-Based Software. Technical Report, UCI-ICS-97-34, University of California, Irvine, August 1997.

[SRW97] J. Stafford, D. J. Richardson and A. L. Wolf. Chaining: A Dependence analysis technique for software architecture. September 1997.

[RH96] G. Rothermell and M. Harrold. Analyzing regression test selection techniques. IEEE Transactions on Software Engineering, 22(8):529-551, August 1996.

[SG96] M. Shaw and D. Garlan. Software Architecture: Perspectives on an Emerging Discipline. Prentice-Hall, Englewood Cliffs, New Jersey, 1996.

[Ves96] S. Vestal. MetaH Programmer's Manual, Version 1.09. Technical Report, Honeywell Technology Center, April 1996.

[Wol97] A. L. Wolf. Succeedings of the Second International Software Architecture Workshop. ACM SIGSOFT Software Engineering Notes, 22( 1): 42-56, January 1997.