An empirical investigation of scenarios gained and lost in architecture evaluation meetings

ABSTRACT

Studying the effectiveness of scenario development meetings in the software architecture process is important to improve meeting effectiveness. This paper reports initial findings from analyzing the data collected in a controlled experiment aimed at studying the effectiveness of meetings in terms of gained and lost scenarios of individuals, real and nominal (non-communicating) teams. Our findings question the effectiveness of holding meetings since more important scenarios were lost than gained in these meetings. In the study nominal teams performed better than individuals and real teams.

REFERENCES

• 1: Muhammed Ali Babar , Stefan Biffl, Eliciting better quality architecture evaluation scenarios: a controlled experiment on top-down vs. bottom-up, Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering, September 21-22, 2006, Rio de Janeiro, Brazil [doi>10.1145/1159733.1159779]
• 2: Muhammad Ali Babar , Barbara Kitchenham , Ross Jeffery, Comparing distributed and face-to-face meetings for software architecture evaluation: A controlled experiment, Empirical Software Engineering, v.13 n.1, p.39-62, February 2008 [doi>10.1007/s10664-007-9052-6]
• 3: PerOlof Bengtsson , Jan Bosch, An experiment on creating scenario profiles for software change, Annals of Software Engineering, v.9 n.1-4, p.59-78, 2000 [doi>10.1023/A:1018968506874]
• 4: Stefan Biffl , Michael Halling, Investigating the Defect Detection Effectiveness and Cost Benefit of Nominal Inspection Teams, IEEE Transactions on Software Engineering, v.29 n.5, p.385-397, May 2003 [doi>10.1109/TSE.2003.1199069]
• 5: Robert P. Biuk-Aghai , Igor T. Hawryszkiewycz, Analysis of Virtual Workspaces, Proceedings of the 1999 International Symposium on Database Applications in Non-Traditional Environments, p.325, November 28-30, 1999
• 6: Fagan, M.: Design and Code Inspections To Reduce Errors In Program Development, IBM Systems J., 1976.
• 7: Kazman R., Barbacci M., Klein M., and Carriere S. J.: Experience with Performing ATAM. Proc. ICSE, 1999.
• 8: Nico Lassing , Perlof Bengtsson , Hans van Vliet , Jan Bosch, Experiences with ALMA: architecture-level modifiability analysis, Journal of Systems and Software, v.61 n.1, p.47-57, March 2002 [doi>10.1016/S0164-1212(01)00113-3]
• 9: Carolyn B. Seaman , Victor R. Basili, Communication and Organization: An Empirical Study of Discussion in Inspection Meetings, IEEE Transactions on Software Engineering, v.24 n.7, p.559-572, July 1998 [doi>10.1109/32.708569]
• 10: Lawrence G. Votta, Jr., Does every inspection need a meeting?, ACM SIGSOFT Software Engineering Notes, v.18 n.5, p.107-114, Dec. 1993
• 11: Winkler D., Biffl S., and Ali Babar M.: An Empirical Study of Scenarios Gained and Lost in Architecture Evaluation Meetings, Technical Report, TU Vienna.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.9 Management
          Subjects: Software quality assurance (SQA)

General Terms:
Experimentation, Management, Measurement

Keywords:
architecture evaluation, quality attributes, scenarios