Keeping the neural networks simple by minimizing the description length of the weights

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Keeping the neural networks simple by minimizing the description length of the weights

Full text

Pdf (889 KB)

Source

Annual Workshop on Computational Learning Theory archive
Proceedings of the sixth annual conference on Computational learning theory table of contents

Santa Cruz, California, United States

Pages: 5 - 13

Year of Publication: 1993

ISBN:0-89791-611-5

Authors

Geoffrey E. Hinton
Drew van Camp

Sponsors

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory
SIGART: ACM Special Interest Group on Artificial Intelligence

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 15, Downloads (12 Months): 47, Citation Count: 21

Additional Information:

references cited by index terms collaborative colleagues

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/168304.168306 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	G E Hinton, Learning translation invariant recognition in massively parallel networks, Volume I: Parallel architectures on PARLE: Parallel Architectures and Languages Europe, p.1-13, March 1987, Eindhoven, The Netherlands
	2	Kevin J. Lang , Alex H. Waibel , Geoffrey E. Hinton, A time-delay neural network architecture for isolated word recognition, Neural Networks, v.3 n.1, p.23-43, 1990 [doi>10.1016/0893-6080(90)90044-L]
	3	Le Cun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W. and Jackel, L. D. (1989) Back-Propagation Applied to Handwritten Zipcode Recognition. Neural Computation, 1, 541-551.
	4	David J. C. MacKay, A practical Bayesian framework for backpropagation networks, Neural Computation, v.4 n.3, p.448-472, May 1992 [doi>10.1162/neco.1992.4.3.448]
	5	Neal, R. M. (1993) Bayesian learning via stochastic dynamics. In Giles, C. L., Hanson, S. J. and Cowan, J.
	6	D. (Eds), Advances in Neural Information Processing Systems 5, Morgan Kaufmann, San Mateo CA.
	7	Steven J. Nowlan , Geoffrey E. Hinton, Simplifying neural networks by soft weight-sharing, Neural Computation, v.4 n.4, p.473-493, July 1992 [doi>10.1162/neco.1992.4.4.473]
	8	Rissanen, J. (1986) Stochastic Complexity and Modeling. Annals of Statistics, 14, 1080-1100.

CITED BY 21

	Harri Valpola , Juha Karhunen, An unsupervised ensemble learning method for nonlinear dynamic state-space models, Neural Computation, v.14 n.11, p.2647-2692, November 2002
	Zoubin Ghahramani, An introduction to hidden Markov models and Bayesian networks, Hidden Markov models: applications in computer vision, World Scientific Publishing Co., Inc., River Edge, NJ, 2001
	Zhe Chen , Simon Haykin, On different facets of regularization theory, Neural Computation, v.14 n.12, p.2791-2846, December 2002
	Tom Heskes , Bart Bakker , Bert Kappen, Approximate algorithms for neural-Bayesian approaches, Theoretical Computer Science, v.287 n.1, p.219-238, 25 September 2002
	Harri Valpola , Markus Harva , Juha Karhunen, Hierarchical models of variance sources, Signal Processing, v.84 n.2, p.267-282, February 2004
	Ricardo Vigário , Andreas Ziehe , Klaus-Robert Müller , Gerd Wübbeler , Lutz Trahms , Bruno-Marcel Mackert , Gabriel Curio , Veikko Jousmäki , Jaakko Särelä , Erkki Oja, Blind decomposition of multimodal evoked responses and DC fields, Exploratory analysis and data modeling in functional neuroimaging, MIT Press, Cambridge, MA, 2003
	Michael I. Jordan , Zoubin Ghahramani , Tommi S. Jaakkola , Lawrence K. Saul, An Introduction to Variational Methods for Graphical Models, Machine Learning, v.37 n.2, p.183-233, Nov.1.1999
	R. A. Choudrey , S. J. Roberts, Variational mixture of Bayesian independent component analyzers, Neural Computation, v.15 n.1, p.213-252, January 2003
	Antti Honkela , Harri Valpola , Juha Karhunen, Accelerating Cyclic Update Algorithms for Parameter Estimation by Pattern Searches, Neural Processing Letters, v.17 n.2, p.191-203, April 2003
	Nicolás García-Pedrajas , Domingo Ortiz-Boyer , César Hervás-Martínez, An alternative approach for neural network evolution with a genetic algorithm: crossover by combinatorial optimization, Neural Networks, v.19 n.4, p.514-528, May 2006
	Gursel Serpen, Hopfield Network as Static Optimizer: Learning the Weights and Eliminating the Guesswork, Neural Processing Letters, v.27 n.1, p.1-15, February 2008
	Markus Harva, 2007 Special Issue: A variational EM approach to predictive uncertainty, Neural Networks, v.20 n.4, p.550-558, May, 2007
	David J. C. MacKay, Comparison of approximate methods for handling hyperparameters, Neural Computation, v.11 n.5, p.1035-1068, July 1, 1999
	Vicente Ruiz De Angulo , Carme Torras, Architecture-Independent Approximation of Functions, Neural Computation, v.13 n.5, p.1119-1135, May 2001
	Shinichi Nakajima , Sumio Watanabe, Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance, Neural Computation, v.19 n.4, p.1112-1153, April 2007
	Tapani Raiko , Harri Valpola , Markus Harva , Juha Karhunen, Building Blocks for Variational Bayesian Learning of Latent Variable Models, The Journal of Machine Learning Research, 8, p.155-201, 5/1/2007
	Tommi S. Jaakkola , Michael I. Jordan, Bayesian parameter estimation via variational methods, Statistics and Computing, v.10 n.1, p.25-37, January 2000
	Guozhong An, The effects of adding noise during backpropagation training on a generalization performance, Neural Computation, v.8 n.3, p.643-674, April 1, 1996
	Ruslan Salakhutdinov , Andriy Mnih, Bayesian probabilistic matrix factorization using Markov chain Monte Carlo, Proceedings of the 25th international conference on Machine learning, p.880-887, July 05-09, 2008, Helsinki, Finland
	Kazuho Watanabe , Motoki Shiga , Sumio Watanabe, Upper bound for variational free energy of Bayesian networks, Machine Learning, v.75 n.2, p.199-215, May 2009
	Shinichi Nakajima , Sumio Watanabe, Generalization error of linear neural networks in an empirical bayes approach, Proceedings of the 19th international joint conference on Artificial intelligence, p.804-810, July 30-August 05, 2005, Edinburgh, Scotland