A Bayesian network classifier with inverse tree structure for voxelwise magnetic resonance image analysis

ABSTRACT

We propose a Bayesian-network classifier with inverse-tree structure (BNCIT) for joint classification and variable selection. The problem domain of voxelwise magnetic-resonance image analysis often involves millions of variables but only dozens of samples. Judicious variable selection may render classification tractable, avoid over-fitting, and improve classifier performance. BNCIT embeds the variable-selection process within the classifier-training process, which makes this algorithm scalable. BNCIT is based on a Bayesian-network model with inverse-tree structure, i.e., the class variable C is a leaf node, and predictive variables are parents of C; thus, the classifier-training process returns a parent set for C, which is a subset of the Markov blanket of C. BNCIT uses voxels in the parent set, and voxels that are probabilistically equivalent to them, as variables for classification of new image data. Since the data set has a limited number of samples, we use the jackknife method to determine whether the classifier generated by BNCIT is a statistical artifact. In order to enhance stability and improve classification accuracy, we model the state of the probabilistically equivalent voxels with a latent variable. We employ an efficient method for determining states of hidden variables, thus reducing dramatically the computational cost of model generation. Experimental results confirm the accuracy and efficiency of BNCIT.

REFERENCES

• 1: Baltimore longitudinal study of aging home page: http://www.grc.nia.nih.gov/branches/blsa/blsa.htm.
• 2: J. Besag. Spatial interaction and statistical analysis of lattice systems. J. Royal Statistical Soc. Series B, 2, 1974.
• 3: John Binder , Daphne Koller , Stuart Russell , Keiji Kanazawa, Adaptive Probabilistic Networks with Hidden Variables, Machine Learning, v.29 n.2-3, p.213-244, Nov./Dec. 1997 [doi>10.1023/A:1007421730016]
• 4: R. R. Bouckaert. Properties of Bayesian network learning algorithms. In R. L. de Mantaras and D. Poole, editors, Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pages 102--109. Morgan Kaufmann, 1994.
• 5: Wray Buntine, A Guide to the Literature on Learning Probabilistic Networks from Data, IEEE Transactions on Knowledge and Data Engineering, v.8 n.2, p.195-210, April 1996 [doi>10.1109/69.494161]
• 6: R. Chen and E. H. Herskovits. Graphical-model based morphometric analysis. accepted, IEEE Trans. Medical Imaging, 2004.
• 7: J. Cheng and R. Greiner. Comparing Bayesian network classifiers. In Proceedings of UAI-99, pages 101--108, 1999.
• 8: Jie Cheng , Russell Greiner, Learning Bayesian Belief Network Classifiers: Algorithms and System, Proceedings of the 14th Biennial Conference of the Canadian Society on Computational Studies of Intelligence: Advances in Artificial Intelligence, p.141-151, June 07-09, 2001
• 9: David Maxwell Chickering, Optimal structure identification with greedy search, The Journal of Machine Learning Research, 3, p.507-554, 3/1/2003 [doi>10.1162/153244303321897717]
• 10: Gregory F. Cooper , Edward Herskovits, A Bayesian Method for the Induction of Probabilistic Networks from Data, Machine Learning, v.9 n.4, p.309-347, Oct. 1992 [doi>10.1023/A:1022649401552]
• 11: Robert Cowell, Introduction to inference for Bayesian networks, Proceedings of the NATO Advanced Study Institute on Learning in graphical models, p.9-26, March 1998, Erice, Italy
• 12: C. Davatzikos. Mapping of image data to stereotaxic spaces: Application to brain mapping. Hum. Brain mapp, 19:334--338, 1998.
• 13: C. Davatzikos, M. Vaillant, S. Resnick, J. Prince, S. Letovsky, and R. Bryan. A computerized method for morphological analysis of the corpus callosum. J. of Comp. Ass. Tomography, (20):88--97, 1996.
• 14: Pedro Domingos , Michael Pazzani, On the Optimality of the Simple Bayesian Classifier under Zero-One Loss, Machine Learning, v.29 n.2-3, p.103-130, Nov./Dec. 1997 [doi>10.1023/A:1007413511361]
• 15: Richard O. Duda , Peter E. Hart , David G. Stork, Pattern Classification (2nd Edition), Wiley-Interscience, 2000
• 16: Nir Friedman , Dan Geiger , Moises Goldszmidt, Bayesian Network Classifiers, Machine Learning, v.29 n.2-3, p.131-163, Nov./Dec. 1997 [doi>10.1023/A:1007465528199]
• 17: S. Geman and D. Geman. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE. Trans. PAMI, 6:721--741, 1984.
• 18: A. Goldszal, C. Davatzikos, D. Pham, M. Yan, R. Bryan, and S. M. Resnick. An image processing protocol for quanlitative and quantitative volumetric analysis of brain images. J. Comput. Assisted Tomogr, 22:827--837, 1998.
• 19: Isabelle Guyon , André Elisseeff, An introduction to variable and feature selection, The Journal of Machine Learning Research, 3, 3/1/2003
• 20: David Heckerman, Bayesian networks for knowledge discovery, Advances in knowledge discovery and data mining, American Association for Artificial Intelligence, Menlo Park, CA, 1996
• 21: David Heckerman, A tutorial on learning with Bayesian networks, Proceedings of the NATO Advanced Study Institute on Learning in graphical models, p.301-354, March 1998, Erice, Italy
• 22: Edward Herskovits, Computer-based probabilistic-network construction, Stanford University, Stanford, CA, 1992
• 23: E. H. Herskovits, H. Peng, and C. Davatzikos. A Bayesian morphometry algorithm. IEEE Trans. Medical Imaging, 23, June 2004.
• 24: Finn V. Jensen , F.V. V. Jensen , F. V. Jensen, Introduction to Bayesian Networks, Springer-Verlag New York, Inc., Secaucus, NJ, 1996
• 25: Ron Kohavi , George H. John, Wrappers for feature subset selection, Artificial Intelligence, v.97 n.1-2, p.273-324, Dec. 1997 [doi>10.1016/S0004-3702(97)00043-X]
• 26: D. Koller and M. Sahami. Towards optimal feature selection. In Proceedings of the 13th International Conference on Machine Learning (ML), pages 284--292, 1996.
• 27: W. Lam and F. Bacchus. Learning Bayesian belief networks: An approach based on the MDL principle. Computational Intelligence, 10:262--293, 1994.
• 28: P. Langley, W. Iba, and K. Thompson. An analysis of Bayesian classifiers. In Proceedings of the Tenth National Conference on Artificial Intelligence, pages 223--228, 1992.
• 29: Judea Pearl, Probabilistic reasoning in intelligent systems: networks of plausible inference, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1988
• 30: I. Rish. An empirical study of the naive Bayes classifier. In IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, 2001.
• 31: D. G. Shen and C. Davatzikos. Hammer: Hierarchical attribute matching mechanism for elastic registration. IEEE Trans. on Medical Imaging, pages 1421--1439, 2002.
• 32: J. Suzuki. A construction of Bayesian networks from databases based on an MDL scheme. In D. Heckerman and A. Mamdani, editors, Proceedings of the Ninth Conference on Uncertainty in Artificial Intelligence, pages 266--273. Morgam Kaufmann, 1993.
• 33: B. Thiesson. Accelerated quantification of Bayesian networks with incomplete data. In Proceedings of the First International Conference on Knowledge Discovery and Data Mining, pages 306--311. AAAI Press, 1995.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Multivariate statistics

Additional Classification:
  J. Computer Applications
  J.3 LIFE AND MEDICAL SCIENCES
      Subjects: Medical information systems

General Terms:
Algorithms

Keywords:
Bayesian network, Markov blanket, classifier, magnetic resonance image