A customizable multi-agent system for distributed data mining

ABSTRACT

We present a general Multi-Agent System framework for distributed data mining based on a Peer-to-Peer model. Agent protocols are implemented through message-based asynchronous communication. The framework adopts a dynamic load balancing policy that is particularly suitable for irregular search algorithms. A modular design allows a separation of the general-purpose system protocols and software components from the specific data mining algorithm. The experimental evaluation has been carried out on a parallel frequent subgraph mining algorithm, which has shown good scalability performances.

REFERENCES

• 1: Rakesh Agrawal , Tomasz Imieliński , Arun Swami, Mining association rules between sets of items in large databases, Proceedings of the 1993 ACM SIGMOD international conference on Management of data, p.207-216, May 25-28, 1993, Washington, D.C., United States
• 2: M. R. Berthold, N. Cebron, F. Dill, G. Di Fatta, T. Gabriel, F. Georg, T. Meinl, P. Ohl, C. Sieb, B. Wiswedel. Knime: the Konstanz Information Miner. Proc. of the 4th Annual Industrial Simulation Conference (ISC), Workshop on Multi-Agent Systems and Simulation (MAS&S), Palermo, Italy, June 5--7, 2006.
• 3: Christian Borgelt , Michael R. Berthold, Mining Molecular Fragments: Finding Relevant Substructures of Molecules, Proceedings of the 2002 IEEE International Conference on Data Mining (ICDM'02), p.51, December 09-12, 2002
• 4: G. Di Fatta and M. R. Berthold. Distributed mining of molecular fragments. Proc. of the Workshop on Data Mining and the Grid (DM-Grid) of the IEEE International Conference on Data Mining (ICDM-04), Nov. 1--4, 2004.
• 5: Giuseppe Di Fatta , Michael R. Berthold, Dynamic Load Balancing for the Distributed Mining of Molecular Structures, IEEE Transactions on Parallel and Distributed Systems, v.17 n.8, p.773-785, August 2006 [doi>10.1109/TPDS.2006.101]
• 6: Demsar J., Zupan B., Leban G. Orange: From Experimental Machine Learning to Interactive Data Mining. White Paper, Faculty of Computer and Information Science, University of Ljubljana, 2004.
• 7: M. Deshpande, M. Kuramochi, and G. Karypis. Automated approaches for classifying structures. Proceedings of Workshop on Data Mining in Bioinformatics (BioKDD), pages 11--18, 2002.
• 8: Giancarlo Fortino , Franco Frattolillo , Wilma Russo , Eugenio Zimeo, Mobile Active Object for Highly Dynamic Distributed Computing, Proceedings of the 16th International Parallel and Distributed Processing Symposium, p.211, April 15-19, 2002
• 9: Giancarlo Fortino , Wilma Russo, Multi-coordination of mobile agents: a model and a component-based architecture, Proceedings of the 2005 ACM symposium on Applied computing, March 13-17, 2005, Santa Fe, New Mexico [doi>10.1145/1066677.1066779]
• 10: M. Klusch, S. Lodi, G. Moro. Agent-based Distributed Data Mining: The KDEC Scheme. Intelligent Information Agents -- The AgentLink Perspective. Lecture Notes in Computer Science 2586 Springer 2003.
• 11: Luck, M., McBurney, P., and Preist, C. Agent technology: enabling next generation computing: A roadmap for agent-based computing. AgentLink Report, 2003. Available from www.agentlink.org/roadmap.
• 12: Ingo Mierswa , Michael Wurst , Ralf Klinkenberg , Martin Scholz , Timm Euler, YALE: rapid prototyping for complex data mining tasks, Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, August 20-23, 2006, Philadelphia, PA, USA [doi>10.1145/1150402.1150531]
• 13: Weislow, O., Kiser, R., Fine, D., Bader, J., Shoemaker, R., Boyd, M. New soluble formazan assay for hiv-1 cytopathic effects: Application to high flux screening of synthetic and natural products for aids antiviral activity. Journal of the National Cancer Institute, University Press, Oxford, UK, 81 (1989), pages 577--586.
• 14: Ian H. Witten , Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 15: Michael Woolridge , Michael J. Wooldridge, Introduction to Multiagent Systems, John Wiley & Sons, Inc., New York, NY, 2001
• 16: Y. Xing, M. G. Madden, J. Duggan, G. Lyons. A Multi-Agent System for Context-based Distributed Data Mining. Technical Report Number NUIG-IT-170503, Department of Information Technology, NUI, Galway, 2003.
• 17: Xifeng Yan , Jiawei Han, gSpan: Graph-Based Substructure Pattern Mining, Proceedings of the 2002 IEEE International Conference on Data Mining (ICDM'02), p.721, December 09-12, 2002
• 18: Mohammed J. Zaki, Parallel and Distributed Association Mining: A Survey, IEEE Concurrency, v.7 n.4, p.14-25, October 1999 [doi>10.1109/4434.806975]
• 19: M. Zaki, S. Parthasarathy, M. Ogihara, and W. Li. New algorithms for fast discovery of association rules. Proceedings of 3rd Int. Conf. on Knowledge Discovery and Data Mining (KDD'97), pages 283--296, 1997.
• 20: Franco Zambonelli , Nicholas R. Jennings , Michael Wooldridge, Developing multiagent systems: The Gaia methodology, ACM Transactions on Software Engineering and Methodology (TOSEM), v.12 n.3, p.317-370, July 2003 [doi>10.1145/958961.958963]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications

General Terms:
Algorithms, Design, Performance

Keywords:
agent oriented software engineering, distributed data mining, dynamic load balancing, multi-agent systems, peer-to-peer computing