Accumulation and presentation of empirical evidence: problems and challenges

ABSTRACT

Understanding the effects of software engineering techniques and processes under varying conditions can be seen as a major prerequisite towards predictable project planning and guaranteeing software quality. Evidence regarding the effects of techniques and processes for specific contexts can be gained by empirical studies. Due to the fact that software development is a human-based and context-oriented activity the effects vary from project environment to project environment. As a consequence, the studies need to be performed in specific environments and the results are typically only valid for these local environments. Potential users of the evidence gained in such studies (e.g., project planners who need to select techniques and processes for a project) are confronted with difficulties such as finding and understanding the relevant results and assessing whether and how they can be applied to their own situation. Thereby, effective transfer and use of empirical findings is hindered. Our thesis is that effective dissemination and exploitation of empirical evidence into industry requires aggregation, integration, and adequate stakeholder-oriented presentation of the results. This position paper sketches major problems and challenges and proposes research issues towards solving the problem.

REFERENCES

• 1: Andreas Jedlitschka , Marcus Ciolkowski, Towards Evidence in Software Engineering, Proceedings of the 2004 International Symposium on Empirical Software Engineering (ISESE'04), p.261-270, August 19-20, 2004 [doi>10.1109/ISESE.2004.31]
• 2: Will Hayes, Research Synthesis in Software Engineering: A Case for Meta-Analysis, Proceedings of the 6th International Symposium on Software Metrics, p.143, November 04-06, 1999
• 3: Barbara A. Kitchenham , Tore Dyba , Magne Jorgensen, Evidence-Based Software Engineering, Proceedings of the 26th International Conference on Software Engineering, p.273-281, May 23-28, 2004
• 4: James Miller, Can Results from Software Engineering Experiments be Safely Combined?, Proceedings of the 6th International Symposium on Software Metrics, p.152, November 04-06, 1999
• 5: Vegas, S.; Juristo, N.; Basili, V.: A Process for Identifying Relevant Information for a Repository: A Case Study for Testing Techniques;, In Aurum, A.; Jeffery, R.; Wohlin, C.; Handzic, M. (Eds): Managing Software Engineering Knowledge; Springer-Verlag; Berlin 2003, pp. 199--230
• 6: Claes Wohlin , Håkan Petersson , Aybüke Aurum, Combining data from reading experiments in software inspections: a feasibility study, Lecture notes on empirical software engineering, World Scientific Publishing Co., Inc., River Edge, NJ, 2003

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

General Terms:
Experimentation

Keywords:
accumulation, aggregation, dissemination, empirical studies, evidence, experience management, exploitation, measurement, software process, software project management