A theory of term weighting based on exploratory data analysis

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

A theory of term weighting based on exploratory data analysis

Full text

Pdf (1.21 MB)

Source

Annual ACM Conference on Research and Development in Information Retrieval archive
Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval table of contents

Melbourne, Australia

Pages: 11 - 19

Year of Publication: 1998

ISBN:1-58113-015-5

Author

Warren R. Greiff

Computer Science Department, University of Massachusetts, Amherst

Sponsors

University of Melbourne : University of Melbourne
SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 3, Downloads (12 Months): 69, Citation Count: 19

Additional Information:

references cited by index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/290941.290948 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	J. P. CaUan, W. B. Croft, and S. M. Harding. The inquery retrieval system. In Proceedings of the 3rd International Conference on Database and Expert Systems Applications, pages 78-83, 1992.
	2	Kenneth Church, William Gale, Patrick Hanks, and Donald Hindle. Using statistics in lexical analysis. In Uri Zernik, editor, Lexical Acquisition: Exploiting On-Line Resources to Build a Lexicon, pages 115- 164, Hillsdale, NJ, 1991. Lawrence Erlbaum Associates.
	3	William S. Cooper , Fredric C. Gey , Daniel P. Dabney, Probabilistic retrieval based on staged logistic regression, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.198-210, June 21-24, 1992, Copenhagen, Denmark [doi>10.1145/133160.133199]
	4	Win. S. Cooper, Aitao Chen, and Fredric C. Gey. Full text retrieval based on probabilistic equations with coefficients fitted by logistic regression. In D. K. Harman, editor, The Second Text REtreival Conference (TREC-2), pages 57-66, Gaithersburg, Md., March 1994. NIST Special Publication 500-215.
	5	W. B. Croft and D. J. Harper. Using probabilistic models of document retrieval without relevance information. Journal of Documentation, 35(4):285- 295, December 1979.
	6	W. B. Croft and Jinxi Xu. Corpus-specific stemming using word form co-occurence. In Proceedings for the Fourth Annual Symposium on Document Analysis and Information Retrieval, pages 147-159, Las Vegas, Nevada, April 1995.
	7	Robert M. Fano. Transmission of Information; a Statistical Theory of Communications. MIT Press, Cambridge, MA, 1961.
	8	Norbert Fuhr, Optimum polynomial retrieval functions based on the probability ranking principle, ACM Transactions on Information Systems (TOIS), v.7 n.3, p.183-204, July 1989 [doi>10.1145/65943.65944]
	9	Norbert Fuhr , Chris Buckley, A probabilistic learning approach for document indexing, ACM Transactions on Information Systems (TOIS), v.9 n.3, p.223-248, July 1991 [doi>10.1145/125187.125189]
	10	Fredric C. Gey, Inferring probability of relevance using the method of logistic regression, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.222-231, July 03-06, 1994, Dublin, Ireland
	11	I. J. Good. Probability and the Weighing of Evidence. Charles Griffin, London, 1950.
	12	I. J. Good. Weight of evidence: A brief survey. In J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith, editors, Bayesian Statistics 2, pages 249-269. North-Holland, Amsterdam, 1983.
	13	Donna Harman. Overview of the first Text REtrieval Conference (TREC-1). In D. K. Harman, editor, The First Text REtrieval Conference (TRECI), pages 1- 20, Gaithersburg, Md., February 1993. NIST Special Publication 500-207.
	14	Donna Harman. Overview of the fifth Text RE- trieval Conference (TREC-5). In E. M. Voorhees and D. K. Harman, editors, The Fifth Text RE- treival Conference (TREC-5), pages 1-28, Gaithersburg, Md. 500-238, November 1997. NIST Special Publication 500-238.
	15	D. J. Harper and C. J. van Rijsbergen. An evaluation of feedback in document retrieval using co-occurrence data. Journal of Documentation, 34(3):189-216, September 1978.
	16	Frederick Hartwig and Brian E. Dearing. Exploratory Data Analysis. Sage Publications, 1979.
	17	S. E. Robertson. Term specificity. Journal of Documentation, 28(2):164-165, 1972. Letter to the editor, with response by K. Sparck Jones.
	18	S. E. Robertson and K. Sparck Jones. Relevance weighting of search terms. Journal of the American Society for Information Science, 27:129-146, 1977.
	19	S. E. Robertson , S. Walker, On relevance weights with little relevance information, Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval, p.16-24, July 27-31, 1997, Philadelphia, Pennsylvania, United States
	20	G. Salton, A. Wong, and C. T. Yu. Automatic indexing using term discrimination and term precision measurements. Information Processing #J Management, 12:43-51, 1976.
	21	G. Salton, H. Wu, and C. Y. Yu. The measurement of term importance in automatic indexing. Journal of the American Society for Information Science, 32:175-186, 1981.
	22	Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
	23	K. Sparck-Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28:11-21, 1972.
	24	John W. Tukey. Exploratory Data Analysis. Addison-Wesley Publishing Company, Reading, MA, 1977.
	25	C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
	26	S. K. M. Wong and Y. Y. Yao. An informationtheoretic measure of term specificity. Journal of the American Society for Information, Science, 43(1):54- 61, 1992.
	27	C. T. Yu , K. Lam , G. Salton, Term Weighting in Information Retrieval Using the Term Precision Model, Journal of the ACM (JACM), v.29 n.1, p.152-170, Jan. 1982 [doi>10.1145/322290.322300]
	28	C. T. Yu , H. Mizuno, Two learning schemes in information retrieval, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.201-218, May 1988, Grenoble, France [doi>10.1145/62437.62451]

CITED BY 19

	Jinxi Xu , W. Bruce Croft, Improving the effectiveness of information retrieval with local context analysis, ACM Transactions on Information Systems (TOIS), v.18 n.1, p.79-112, Jan. 2000
	Kimberley Deninger , Caroline Cassigneul, The relevance factor, Journal of Computing Sciences in Colleges, v.16 n.4, p.337-340, 2001
	Akiko Aizawa, The feature quantity: an information theoretic perspective of Tfidf-like measures, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.104-111, July 24-28, 2000, Athens, Greece
	Akiko Aizawa, An information-theoretic perspective of tf—idf measures, Information Processing and Management: an International Journal, v.39 n.1, p.45-65, January 2003
	Justin Picard , Jacques Savoy, Enhancing retrieval with hyperlinks: a general model based on propositional argumentation systems, Journal of the American Society for Information Science and Technology, v.54 n.4, p.347-355, February 15, 2003
	Jaime Teevan , David R. Karger, Empirical development of an exponential probabilistic model for text retrieval: using textual analysis to build a better model, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada
	Raj D. Iyer , David D. Lewis , Robert E. Schapire , Yoram Singer , Amit Singhal, Boosting for document routing, Proceedings of the ninth international conference on Information and knowledge management, p.70-77, November 06-11, 2000, McLean, Virginia, United States
	Sumio Fujita, NTCIR-2 as a Rosetta stone in laboratory experiments of IR systems, Information Processing and Management: an International Journal, v.41 n.3, p.489-506, May 2005
	Hui Fang , ChengXiang Zhai, An exploration of axiomatic approaches to information retrieval, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil
	Jin Zhang , Tien N. Nguyen, A New Term Significance Weighting Approach, Journal of Intelligent Information Systems, v.24 n.1, p.61-85, January 2005
	Sumio Fujita, Discriminative power and retrieval effectiveness of phrasal indexing terms, Proceedings of the ACL-2000 workshop on Recent advances in natural language processing and information retrieval: held in conjunction with the 38th Annual Meeting of the Association for Computational Linguistics, October 08-08, 2000, Hong Kong
	Andreas Nürnberger , Aljoscha Klose , Rudolf Kruse, Self-organizing maps for interactive search in document databases, Intelligent exploration of the web, Physica-Verlag GmbH, Heidelberg, Germany, 2003
	Ronan Cummins , Colm O'Riordan, Evolving General Term-Weighting Schemes for Information Retrieval: Tests on Larger Collections, Artificial Intelligence Review, v.24 n.3-4, p.277-299, November 2005
	Dietmar Wolfram , Jin Zhang, The influence of indexing practices and weighting algorithms on document spaces, Journal of the American Society for Information Science and Technology, v.59 n.1, p.3-11, January 2008
	Ronan Cummins , Colm O'Riordan, Evolving local and global weighting schemes in information retrieval, Information Retrieval, v.9 n.3, p.311-330, June 2006
	Kristof Coussement , Dirk Van den Poel, Integrating the voice of customers through call center emails into a decision support system for churn prediction, Information and Management, v.45 n.3, p.164-174, April, 2008
	Lillian Lee, IDF revisited: a simple new derivation within the Robertson-Spärck Jones probabilistic model, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands
	Ping Li, Very sparse stable random projections for dimension reduction in lα (0 <α ≤ 2) norm, Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, August 12-15, 2007, San Jose, California, USA
	Donald Metzler, Generalized inverse document frequency, Proceeding of the 17th ACM conference on Information and knowledge management, October 26-30, 2008, Napa Valley, California, USA