Boosting as entropy projection

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Boosting as entropy projection

Full text

Pdf (1.26 MB)

Source

Annual Workshop on Computational Learning Theory archive
Proceedings of the twelfth annual conference on Computational learning theory table of contents

Santa Cruz, California, United States

Pages: 134 - 144

Year of Publication: 1999

ISBN:1-58113-167-4

Authors

Jyrki Kivinen	Department of Computer Science, P.O. Box 26 (Teollisuuskatu 23), FIN-00014 University of Helsinki, Finland
Manfred K. Warmuth	Computer Science Department, University of California, Santa Cruz, Santa Cruz, CA

Sponsors

SIGACT: ACM Special Interest Group on Algorithms and Computation Theory
SIGART: ACM Special Interest Group on Artificial Intelligence
Univ. of California, : University of California at Santa Cruz

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 10, Downloads (12 Months): 42, Citation Count: 24

Additional Information:

references cited by index terms collaborative colleagues

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/307400.307424 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	Ama85	S. Amari. Differential Geometrical Methods in Statistics. Springer, Berlin, 1985.
	AW99	K. Azoury and M. K. Warmuth. Relative loss bounds for on-line density estimation with the exponential family of distributions. In Proc. 15th Conf. on Uncertainty in Artificial Intelligence. Morgan Kaufmann, San Francisco, CA, 1999.
	Bre67	L.M. Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Physics, 7:200-217, 1967.
	BSS93	M.S. Razaraa, H. D. Shera!i, and C. M. Shetty. Nonlinear Programming: Theory and Algorithms. Wiley, New York, NY, 1993. Second edition.
	Byl97	Tom Bylander, The binary exponentiated gradient algorithm for learning linear functions, Proceedings of the tenth annual conference on Computational learning theory, p.184-192, July 06-09, 1997, Nashville, Tennessee, United States [doi>10.1145/267460.267495]
	CBKW94	N. Cesa-Bianchi, A. Krogh, and M. K. Warmath. Bounds on approximate steepest de- ~IIL 1U{ III~elIIIUUU IIIaXIIIIILaLIOII 111 eApUtl~iitial families. IEEE Transaction on Information Theory, 40(4): 1215-1220, 1994.
	CL81	Y. Censor and A. Lent. An iterative row-action method for interval convex programming. Journal of Optimization Theory and Applications, 34(3):321-353, 1981.
	Csi91	I. Csiszar. Why least squares and maximum entropy? An axiomatic approach for linear inverse problems. The Annals of Statistics, 19(4):2032- 2066, 1991.
	DDL97	Stephen Della Pietra , Vincent Della Pietra , John Lafferty, Inducing Features of Random Fields, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.4, p.380-393, April 1997 [doi>10.1109/34.588021]
	DH99	Nigel Duffy , David P. Helmbold, A Geometric Approach to Leveraging Weak Learners, Proceedings of the 4th European Conference on Computational Learning Theory, p.18-33, March 29-31, 1999
	FHT98	J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Technical report, Stanford University, 1998.
	Fre95	Yoav Freund, Boosting a weak learning algorithm by majority, Information and Computation, v.121 n.2, p.256-285, Sept. 1995 [doi>10.1006/inco.1995.1136]
	FS97a	Y.Freund and R.E. Schapire. Adaptive game playing using multiplicative weights. To appear in Games and Economic Behavior.
	FS97b	Yoav Freund , Robert E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences, v.55 n.1, p.119-139, Aug. 1997 [doi>10.1006/jcss.1997.1504]
	FSSW97	Yoav Freund , Robert E. Schapire , Yoram Singer , Manfred K. Warmuth, Using and combining predictors that specialize, Proceedings of the twenty-ninth annual ACM symposium on Theory of computing, p.334-343, May 04-06, 1997, El Paso, Texas, United States [doi>10.1145/258533.258616]
	HUL91	J.-B. Hiriart-Urruty and C.Lemarechal. Con vex Analysis and Minimization Algorithms II. Springer, Berlin, 1991.
	HW98	Mark Herbster , Manfred K. Warmuth, Tracking the best regressor, Proceedings of the eleventh annual conference on Computational learning theory, p.24-31, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279949]
	JB90	L. Jones and C. Byrne. General entropy criteria for inverse problems, with applications to data compression, pattern classification and cluster analysis. IEEE Transactions on Information Theory, 36(1):23-30, 1990.
	Jum90	G. Jumarie, Relative information: theories and applications, Springer-Verlag New York, Inc., New York, NY, 1990
	KK92	J.N. Kapur and H. K Kesavan. Entropy Optimization Principles with Applications. Academic Press, 1992.
	KW97	Jyrki Kivinen , Manfred K. Warmuth, Exponentiated gradient versus gradient descent for linear predictors, Information and Computation, v.132 n.1, p.1-63, Jan. 10, 1997 [doi>10.1006/inco.1996.2612]
	KW99	Jyrki Kivinen , Manfred K. Warmuth, Averaging Expert Predictions, Proceedings of the 4th European Conference on Computational Learning Theory, p.153-167, March 29-31, 1999
	Laf99	John Lafferty, Additive models, boosting, and inference for generalized divergences, Proceedings of the twelfth annual conference on Computational learning theory, p.125-133, July 07-09, 1999, Santa Cruz, California, United States [doi>10.1145/307400.307422]
	LLW92	Nicholas Littlestone , Philip M. Long , Manfred K. Warmuth, On-line learning of linear functions, Computational Complexity, v.5 n.1, p.1-23, Jan. 1995 [doi>10.1007/BF01277953]
	Lue84	D.G. Luenberger. Linear and Nonlinear Programming. Addison-Wesley, Reading, MA, 1984. Second edition.
	Roc70	R. Rockafellar. Convex Analysis. Princeton University Press, 1970.
	ROM98	G. Ratch, T. Onoda, and K. Miiller. Soft margins for AdaBoost. Technical Report NC-TR- 1998-021, NeuroCOLT2 Technical Report Series, 1998.
	Sch90	Robert E. Schapire, The Strength of Weak Learnability, Machine Learning, v.5 n.2, p.197-227, Jun. 1990 [doi>10.1023/A:1022648800760]
	SS98	Robert E. Schapire , Yoram Singer, Improved boosting algorithms using confidence-rated predictions, Proceedings of the eleventh annual conference on Computational learning theory, p.80-91, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279960]

CITED BY 24

	John Lafferty, Additive models, boosting, and inference for generalized divergences, Proceedings of the twelfth annual conference on Computational learning theory, p.125-133, July 07-09, 1999, Santa Cruz, California, United States
	Nigel Duffy , David Helmbold, A geometric approach to leveraging weak learners, Theoretical Computer Science, v.284 n.1, p.67-108, 6 July 2002
	Ron Meir , Gunnar Rätsch, An introduction to boosting and leveraging, Advanced lectures on machine learning, Springer-Verlag New York, Inc., New York, NY, 2003
	Philip M. Long, Minimum majority classification and boosting, Eighteenth national conference on Artificial intelligence, p.181-186, July 28-August 01, 2002, Edmonton, Alberta, Canada
	Noboru Murata , Takashi Takenouchi , Takafumi Kanamori , Shinto Eguchi, Information geometry of U-Boost and Bregman divergence, Neural Computation, v.16 n.7, p.1437-1481, July 2004
	Jean-Christophe Janodet , Richard Nock , Marc Sebban , Henri-Maxime Suchier, Boosting grammatical inference with confidence oracles, Proceedings of the twenty-first international conference on Machine learning, p.54, July 04-08, 2004, Banff, Alberta, Canada
	Zhihua Zhang , James T. Kwok , Dit-Yan Yeung, Surrogate maximization/minimization algorithms for AdaBoost and the logistic regression model, Proceedings of the twenty-first international conference on Machine learning, p.117, July 04-08, 2004, Banff, Alberta, Canada
	Gunnar Rätsch , Sebastian Mika , Bernhard Schölkopf , Klaus-Robert Müller, Constructing Boosting Algorithms from SVMs: An Application to One-Class Classification, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.9, p.1184-1199, September 2002
	Philip M. Long , Vinsensius Berlian Vega, Boosting and Microarray Data, Machine Learning, v.52 n.1-2, p.31-44, July-August 2003
	Manfred K. Warmuth , Jun Liao , Gunnar Rätsch, Totally corrective boosting algorithms that maximize the margin, Proceedings of the 23rd international conference on Machine learning, p.1001-1008, June 25-29, 2006, Pittsburgh, Pennsylvania
	Michael Collins , Robert E. Schapire , Yoram Singer, Logistic Regression, AdaBoost and Bregman Distances, Machine Learning, v.48 n.1-3, p.253-285, 2002
	Gunnar Rätsch , Ayhan Demiriz , Kristin P. Bennett, Sparse Regression Ensembles in Infinite and Finite Hypothesis Spaces, Machine Learning, v.48 n.1-3, p.189-218, 2002
	Zhihua Zhang , James T. Kwok , Dit-Yan Yeung, Surrogate maximization/minimization algorithms and extensions, Machine Learning, v.69 n.1, p.1-33, October 2007
	Richard Nock , Frank Nielsen, A Real generalization of discrete AdaBoost, Artificial Intelligence, v.171 n.1, p.25-41, January, 2007
	Arindam Banerjee, On Bayesian bounds, Proceedings of the 23rd international conference on Machine learning, p.81-88, June 25-29, 2006, Pittsburgh, Pennsylvania
	Linlin Shen , Li Bai, MutualBoost learning for selecting Gabor features for face recognition, Pattern Recognition Letters, v.27 n.15, p.1758-1767, November 2006
	Linlin Shen , Li Bai, MutualBoost learning for selecting Gabor features for face recognition, Pattern Recognition Letters, v.27 n.15, p.1758-1767, November, 2006
	Gunnar Rätsch , Manfred K. Warmuth, Efficient Margin Maximizing with Boosting, The Journal of Machine Learning Research, 6, p.2131-2152, 12/1/2005
	Zhihua Zhang , James T. Kwok , Dit-Yan Yeung, Surrogate maximization/minimization algorithms and extensions, Machine Learning, v.69 n.1, p.1-33, October 2007
	Amitava Karmaker , Stephen Kwek, A boosting approach to remove class label noise, International Journal of Hybrid Intelligent Systems, v.3 n.3, p.169-177, August 2006
	Sudipto Guha , Piotr Indyk , Andrew Mcgregor, Sketching information divergences, Machine Learning, v.72 n.1-2, p.5-19, August 2008
	Richard Nock , Frank Nielsen, A Real generalization of discrete AdaBoost, Proceeding of the 2006 conference on ECAI 2006: 17th European Conference on Artificial Intelligence August 29 -- September 1, 2006, Riva del Garda, Italy, p.509-515, May 22, 2006
	Ivor W. Tsang , James T. Kwok, Ensembles of partially trained SWMs with multiplicative updates, Proceedings of the 20th international joint conference on Artifical intelligence, p.1089-1094, January 06-12, 2007, Hyderabad, India
	Qing Wang , Sanjeev R. Kulkarni , Sergio Verdú, Universal Estimation of Information Measures for Analog Sources, Foundations and Trends in Communications and Information Theory, v.5 n.3, p.265-353, March 2009