Coordinating agent activities in knowledge discovery processes

ABSTRACT

Knowledge discovery in databases (KDD) is an increasingly widespread activity. KDD processes may entail the use of a large number of data manipulation and analysis techniques, and new techniques are being developed on an ongoing basis. A challenge for the effective use of KDD is coordinating the use of these techniques, which may be highly specialized, conditional and contingent. Additionally, the understanding and validity of KDD results can depend critically on the processes by which they were derived. We propose to use process programming to address the coordination of agents in the use of KDD techniques. We illustrate this approach using the process language Little-JIL to program a representative bivariate regression process. With Little-JIL programs we can clearly capture the coordination of KDD activities, including control flow, pre- and post-requisites, exception handling, and resource usage.

REFERENCES

• 1: Paul R. Cohen, Empirical methods for artificial intelligence, MIT Press, Cambridge, MA, 1995
• 2: Cleveland, W. S. and Loader, C. R. Smoothing by local regression: principles and methods (with discussion). Computational Statistics, 1995.
• 3: Matthew B. Dwyer , Lori A. Clarke, Data flow analysis for verifying properties of concurrent programs, Proceedings of the 2nd ACM SIGSOFT symposium on Foundations of software engineering, p.62-75, December 06-09, 1994, New Orleans, Louisiana, United States
• 4: Eugene S Edgington, Randomization tests, Marcel Dekker, Inc., New York, NY, 1986
• 5: Emerson, J. D., and Hoaglin, D. C.. Resistant lines for x versus y. In Hoaglin, D. C.; Mosteller, F.; and Tukey, J. W., (Eds), Understanding Robust and Exploratory Data Analysis. Wiley. 129-l 65, 1983.
• 6: Engels, Robert, Guido Lindner, and Rudi Studer. A guided tour through the data mining jungle, in Proc. aE the Third Intl. Conf. on Knowledge Discovery and Data Mining (1997), 163-166.
• 7: Fayyad, Usama, Gregory Piatetsky-Shapiro, and Padhraic Smyth. From data mining to knowledge discovery in databases. AI Magazine. (Fall 1996), 37- 54.
• 8: Feelders, A. and W. Werkooijen. Which method learns the most from data? Methodological issues in the analysis of comparative studies, in Preliminary papers of the Fifth Intl. Workshop on Artificial Intelligence and Statistics (1995).
• 9: Gaines, Brian R. Refining induction into knowledge. Knowledge Discovery in Data Bases, in Workshop Notes from the Ninth Natl. Conf. on Artificial Intelligence (1991). l-10.
• 10: Garvey, Alan, Keith Decker, and Victor Lesser. A Negotiation-Based Interface between a Real-Time Scheduler and a Decision-Maker, in Proc. of Workshop on Models of Conflict Management in Cooperative Problem Solving (1994), AAAI.
• 11: Olivier Gascuel , Gilles Caraux, Statistical significance in inductive learning, Proceedings of the 10th European conference on Artificial intelligence, p.435-439, August 1992, Vienna, Austria
• 12: Giles, C. Lee and Steve Lawrence. Presenting and analyzing the results of AI experiments: Data averaging and data snooping, in Proc. of the Fourteenth Natl. Conf. on Artificial Intelligence (1997), AAAI Press. 362-367.
• 13: Dennis Heimbigner, The ProcessWall: a process state server approach to process programming, Proceedings of the fifth ACM SIGSOFT symposium on Software development environments, p.159-168, December 09-11, 1992, Tyson's Corner, Virginia, United States
• 14: David D. Jensen , Paul R. Cohen, Multiple Comparisons in Induction Algorithms, Machine Learning, v.38 n.3, p.309-338, March 2000 [doi>10.1023/A:1007631014630]
• 15: David Jensen , Tim Oates , Paul R. Cohen, Building Simple Models: A Case Study with Decision Trees, Proceedings of the Second International Symposium on Advances in Intelligent Data Analysis, Reasoning about Data, p.211-222, August 04-06, 1997
• 16: Jensen, D. D. Unique Challenges of Managing Inductive Knowledge, in Working Notes of the AAAI 1997 Spring Symp. on Artificial Intelligence in Knowledge Management (1997), 75-81. Also in: Artificial Intelligence and Knowledge Management. Collected Papers from the 1997 Workshop. Technical Report WS-97-09. AAAI Press. 25-3 1.
• 17: Barbara Staudt Lerner , Leon J. Osterweil , Stanley M. Sutton, Jr. , Alexander E. Wise, Programming Process Coordination in Little-JIL, Proceedings of the 6th European Workshop on Software Process Technology, p.127-131, September 16-18, 1998
• 18: Barbara Staudt Lerner , Stanley M. Sutton Jr , Leon J. Osterweil, Enhancing Design Methods to Support Real Design Processes, Proceedings of the 9th international workshop on Software specification and design, p.159, April 16-18, 1998
• 19: Eric K. McCall , Lori A. Clarke , Leon J. Osterweil, An adaptable generation approach to agenda management, Proceedings of the 20th international conference on Software engineering, p.282-291, April 19-25, 1998, Kyoto, Japan
• 20: Noreen, E.W. Computer Intensive Methods fbr Testing Hypotheses: An Introduction. Wiley, 1989.
• 21: St. Amant, R. and Cohen, P.R. Submitted to Intelligent Support for Exploratory Data Analysis, 1997.
• 22: Stanley M. Sutton, Jr. , Leon J. Osterweil, The design of a next-generation process language, Proceedings of the 6th European conference held jointly with the 5th ACM SIGSOFT international symposium on Foundations of software engineering, p.142-158, September 22-25, 1997, Zurich, Switzerland
• 23: Wagner, T., Garvey, A., and Lesser, V. Complex Goal Criteria and its Application in Design-to- Criteria Scheduling, in Proc. of the Fourteenth Natl. Conf. on Artificial Intelligence (1997).
• 24: Weiss, S. and Kulikowski, C.A. Computer Systems That Learn. Morgan K.aufma.nn, 199 1.
• 25: A. Wise, Little-JIL 1.0 Language Report TITLE2:, University of Massachusetts, Amherst, MA, 1998
• 26: Zhong, N., Liu, C., Kakemoto, Y., and Ohsuga S. KDD process planning, in Proc. of the Third Intl. Conf. on Knowledge Discovery and Data Mining (1997), 291-294.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
  D.3 PROGRAMMING LANGUAGES

  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Correlation and regression analysis

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.4 Knowledge Representation Formalisms and Methods
          Subjects: Representations (procedural and rule-based)

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management
          Subjects: Software process

General Terms:
Design, Languages, Management, Performance, Theory

Keywords:
agenda management, agent coordination, knowledge discovery process, knowledge representation, process programming