Using data mining and recommender systems to scale up the requirements process

ABSTRACT

Ultra-Large-Scale (ULS) software projects are anticipated to be highly complex and to involve thousands, or even hundreds of thousands of stakeholders. Unfortunately numerous accounts of recent failures and challenges in industrial and governmental projects have demonstrated that current requirements elicitation and prioritization practices do not scale adequately to address the needs of large projects. This position paper directly addresses this problem through proposing an open, inclusive, and robust elicitation and prioritization process that utilizes data-mining and recommender technologies to facilitate the active involvement of many thousands of stakeholders. We believe that the approach described in this paper is a fundamental building block towards addressing higher level requirements problems facing ULS Systems.

REFERENCES

• 1: Chumki Basu , Haym Hirsh , William Cohen, Recommendation as classification: using social and content-based information in recommendation, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.714-720, July 1998, Madison, Wisconsin, United States
• 2: Robin Burke , Bamshad Mobasher , Chad Williams , Runa Bhaumik, Detecting Profile Injection Attacks in Collaborative Recommender Systems, Proceedings of the The 8th IEEE International Conference on E-Commerce Technology and The 3rd IEEE International Conference on Enterprise Computing, E-Commerce, and E-Services, p.23, June 26-29, 2006 [doi>10.1109/CEC-EEE.2006.34]
• 3: Jane Cleland-Huang , Raffaella Settimi , Xuchang Zou , Peter Solc, Automated classification of non-functional requirements, Requirements Engineering, v.12 n.2, p.103-120, May 2007 [doi>10.1007/s00766-007-0045-1]
• 4: Alan Davis , Oscar Dieste , Ann Hickey , Natalia Juristo , Ana M. Moreno, Effectiveness of Requirements Elicitation Techniques: Empirical Results Derived from a Systematic Review, Proceedings of the 14th IEEE International Requirements Engineering Conference (RE'06), p.176-185, September 11-15, 2006 [doi>10.1109/RE.2006.17]
• 5: Chuan Duan , Jane Cleland-Huang, A Clustering Technique for Early Detection of Dominant and Recessive Cross-Cutting Concerns, Proceedings of the Early Aspects at ICSE: Workshops in Aspect-Oriented Requirements Engineering and Architecture Design, p.1, May 20-26, 2007 [doi>10.1109/EARLYASPECTS.2007.1]
• 6: Chuan Duan , Jane Cleland-Huang, Clustering support for automated tracing, Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, November 05-09, 2007, Atlanta, Georgia, USA [doi>10.1145/1321631.1321668]
• 7: Duan, C., Clustering and its Application in Requirements Engineering, Technical Report #08-001, School of Computing., DePaul University, Available online at http://www.cs.depaul.edu (Chicago, Feb. 2008).
• 8: Goldstein, H. "Who Killed the Virtual Case File?" IEEE Spectrum, 42,9 (2005) 24--35.
• 9: Gruenbacher, P.,, "Integrating Groupware and CASE Capabilities for Improving Stakeholder Involvement in Requirements Engineering," Euromicro, 2 (2000) 2232,
• 10: Hooks, I. F., & Farry, K. Creating Successful Products Through Smart Requirements Management. New York: Amacon. (2001).
• 11: Joachim Karlsson , Kevin Ryan, A Cost-Value Approach for Prioritizing Requirements, IEEE Software, v.14 n.5, p.67-74, September 1997 [doi>10.1109/52.605933]
• 12: Shyong K. Lam , John Riedl, Shilling recommender systems for fun and profit, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988726]
• 13: N. R. Mead, "Requirements Prioritization Introduction", Software Eng. Inst. web pub., Carnegie Mellon Univ., (2006).
• 14: Mobasher, B., Burke, R., & Sandvig, J., "Model-based collaborative filtering as a defense against profile injection attacks", Proceedings of the 21st National Conference on Artificial Intelligence, (Boston, MA, 2006).
• 15: Bamshad Mobasher , Robin Burke , Runa Bhaumik , Chad Williams, Toward trustworthy recommender systems: An analysis of attack models and algorithm robustness, ACM Transactions on Internet Technology (TOIT), v.7 n.4, p.23-es, October 2007 [doi>10.1145/1278366.1278372]
• 16: Northrop, L., Feiler, P., Gabriel, R., Goodenought, J., Linger, R., Longstaff, T., Kazman, R., Klein, M., Schmidt, D., Sullivan, K., Wallnau, K., Ultra-Large-Scale Systems: The Software Challenge of the Future, Technical Report, Software Engineering Institute, Carnegie Mellon, (June 2006).
• 17: J. F. Nunamaker Jr , R. O. Briggs , D. D. Mittleman, Lessons from a Decade of Group Support Systems ResearchJay F. Nunamaker, Jr., Robert O. Briggs, and Daniel D. Mittleman, Proceedings of the 29th Hawaii International Conference on System Sciences (HICSS) Volume 3: Collaboration Systems and Technology, p.418, January 03-06, 1996
• 18: Pazzani, M., & Billsus, D. "Content-Based Recommendation Systems". In P. Brusilovsky, A. Kobsa, & W. Nejdl, The Adaptive Web: Methods and Strategies of Web Personalization, Berlin Heidelberg NewYork: Springer-Verlag. (2007).
• 19: Bamshad Mobasher , Robin Burke , Runa Bhaumik , J. J. Sandvig, Attacks and Remedies in Collaborative Recommendation, IEEE Intelligent Systems, v.22 n.3, p.56-63, May 2007 [doi>10.1109/MIS.2007.45]
• 20: Schafer, J. B., Frankowski, D., & Shilad, S., "Collaborative Filtering Recommender Systems" In P. Brusilovsky, A. Kobsa, & W. Nejdl, The Adaptive Web: Methods and Strategies of Web Personalization. New York: Springer-Verlag. (2007).
• 21: Karl Eugene Wiegers, Software Requirements, Microsoft Press, Redmond, WA, 1999

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications
          Subjects: Elicitation methods (e.g., rapid prototyping, interviews, JAD)

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering; Information filtering

General Terms:
Algorithms, Human Factors, Management

Keywords:
clustering, data mining, recommender systems, requirements elicitation, requirements prioritization, ultra-large scale systems