A new distributed data mining model based on similarity

ABSTRACT

Distributed Data Mining (DDM) has been very active and enjoying a growing amount attention since its inception. Current DDM techniques regard the distributed data sets as a single virtual table and assume there exists a global model which could be generated if the data were combined/centralized. This paper proposes a similarity-based distributed data mining(SBDDM) framework which explicitly take the differences among distributed sources into consideration. A new similarity measure is introduced and its effectiveness is then evaluated and validated. This paper also illustrates the limitations of current DDM techniques through three concrete case studies. Finally distributed clustering within the SBDDM framework is also discussed.

REFERENCES

• 1: Charu C. Aggarwal , Joel L. Wolf , Philip S. Yu , Cecilia Procopiuc , Jong Soo Park, Fast algorithms for projected clustering, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.61-72, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 2: Rakesh Agrawal , Tomasz Imieliński , Arun Swami, Mining association rules between sets of items in large databases, Proceedings of the 1993 ACM SIGMOD international conference on Management of data, p.207-216, May 25-28, 1993, Washington, D.C., United States
• 3: Douglas Burdick , Manuel Calimlim , Johannes Gehrke, MAFIA: A Maximal Frequent Itemset Algorithm for Transactional Databases, Proceedings of the 17th International Conference on Data Engineering, p.443-452, April 02-06, 2001
• 4: Chan, P. C., & Stolfo, S. (1993). Meta-learning for multistrategy and parallel learning. Proceedings of the Second International Workshop on Multistrategy Learning.
• 5: David W. Cheung , Vincent T. Ng , Ada W. Fu , Yongjian Fu, Efficient Mining of Association Rules in Distributed Databases, IEEE Transactions on Knowledge and Data Engineering, v.8 n.6, p.911-922, December 1996 [doi>10.1109/69.553158]
• 6: Vincent Cho , Beat Wüthrich, Towards Real Time Discovery from Distributed Information Sources, Proceedings of the Second Pacific-Asia Conference on Research and Development in Knowledge Discovery and Data Mining, p.376-377, April 15-17, 1998
• 7: Gautam Das , Dimitrios Gunopulos , Heikki Mannila, Finding Similar Time Series, Proceedings of the First European Symposium on Principles of Data Mining and Knowledge Discovery, p.88-100, June 24-27, 1997
• 8: Gautam Das , Heikki Mannila, Context-Based Similarity Measures for Categorical Databases, Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery, p.201-210, September 13-16, 2000
• 9: Venkatesh Ganti , Johannes Gehrke , Raghu Ramakrishnan, A framework for measuring changes in data characteristics, Proceedings of the eighteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, p.126-137, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States [doi>10.1145/303976.303989]
• 10: Karam Gouda , Mohammed Javeed Zaki, Efficiently Mining Maximal Frequent Itemsets, Proceedings of the 2001 IEEE International Conference on Data Mining, p.163-170, November 29-December 02, 2001
• 11: Kargupta, H., & Chan, P. (Eds.). (2000). Advances in distributed and parallel data mining. AAAI Press.
• 12: Kargupta, H., Park, B., Hershbereger, D., & Johnson, E. (2000). Collective data mining: A new perspective toward distributed data mining. In H. Kargupta and P. Chan (Eds.), Advances in distributed data mining, 133--184. AAAI/MIT.
• 13: W. Lam , A. M. Segre, Distributed data mining of probabilistic knowledge, Proceedings of the 17th International Conference on Distributed Computing Systems (ICDCS '97), p.178, May 27-30, 1997
• 14: Li, T., Ogihara, M., & Zhu, S. (2002). Similarity testing between heterogeneous basket databases (Technical Report 781). Computer Science, Univ. of Rochester.
• 15: Srinivasan Parthasarathy , Mitsunori Ogihara, Clustering Distributed Homogeneous Datasets, Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery, p.566-574, September 13-16, 2000
• 16: R. Wirth, M. B., & Hipp, J. (2001). When distribution is part of the semantics: A new problem class for distributed knowledge discovery. In Proceedings of workshop on Ubiquitous Data Mining for Mobile and Distributed Environments, PKDD/ECML 2001.
• 17: Rafiei, D., & Mendelzon, A. (1997). Similarity-based queries for time series data (pp. 13--25).
• 18: Ronkainen, R. (1998). Attribute similarity and event sequence similarity in data mining. Ph.lic.thesis, University of Helsinki. Available as Report C-1998-42, University of Helsinki, Department of Computer Science, October 1998.
• 19: Subramonian, R. (1998). Defining diff as a data mining primitive. KDD.
• 20: Turnisky, A., & Grossman, R. (2000). A framework for finding distributed data mining strategies that are intermediate between centralized strategies and in-place strategies. Proc. of KDD Workshop on Distributed Data Mining.
• 21: Kenji Yamanishi, Distributed cooperative Bayesian learning strategies, Proceedings of the tenth annual conference on Computational learning theory, p.250-262, July 06-09, 1997, Nashville, Tennessee, United States [doi>10.1145/267460.267507]
• 22: Zaki, M., & Ho, C. (Eds.). (2000). Large-scale parallel data mining. Springer.
• 23: Shenghuo Zhu , Tao Li, An Algorithm for Non-Distance Based Clustering in High Dimensional Spaces, University of Rochester, Rochester, NY, 2002

INDEX TERMS

Keywords:
Distributed Data Mining (DDM), SBDDM, similarity