A probabilistic learning approach for document indexing

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

A probabilistic learning approach for document indexing

Full text

Pdf (1.69 MB)

Source

ACM Transactions on Information Systems (TOIS) archive
Volume 9 , Issue 3 (July 1991) table of contents
Special issue on research and development in information retrieval

Pages: 223 - 248

Year of Publication: 1991

ISSN:1046-8188

Authors

Norbert Fuhr	Univ. Dortmund, Dortmund, Germany
Chris Buckley	Cornell Univ., Ithaca, NY

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 6, Downloads (12 Months): 65, Citation Count: 45

Additional Information:

references cited by index terms reviews collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/125187.125189 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	BEINKE-GEISER, U., LUSTIG, G., AND PUTZE-MEIER, G Indexieren mit dem System DAISY. In Automatische Indexierung zwischen Forschung und Anwendung, G. Lustig, Ed. Olms, Hildesheim, Germany, 1986, pp. 73-97.
	2	P. Biebricher , N. Fuhr , G. Lustig , M. Schwantner , G. Knorz, The automatic indexing system AIR/PHYS - from research to applications, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.333-342, May 1988, Grenoble, France [doi>10.1145/62437.62470]
	3	BIEBRICHER, P., FUHR, N., KNORZ, G., LUSTIG, G., AND SCHWANTNER, M. Entwicklung und Anwendung des automatischen Indexierungssystems AIR/PHYS. Nachrwhten fuer Dokumentation 39 (1988), 135-143.
	4	CHOW, C. K., AND LIU, C N. Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf Theor. 14, 3, (1968), 462-467.
	5	CROFT, W.B. Boolean queries and term dependencies in probabilistic retrieval models. J. Am. Soc. Inf. Sc~. 37, 2 (1986), 71-77.
	6	CROFT, W.B. Document representation in probabilistic models of information retrieval. J. Am. Soc. Inf. Sc~. 32, (1981), 451-457.
	7	CROFT, W.B. Experiments with representation in a document retrieval system Inf. Tech. Res. Dev. 2, (1983), 1-22.
	8	J. Fagan, Automatic phrase indexing for document retrieval, Proceedings of the 10th annual international ACM SIGIR conference on Research and development in information retrieval, p.91-101, June 03-05, 1987, New Orleans, Louisiana, United States [doi>10.1145/42005.42016]
	9	Joel L. Fagan, The effectiveness of a nonsyntatic approach to automatic phrase indexing for document retrieval, Journal of the American Society for Information Science, v.40 n.2, p.115-132, March 1989 [doi>10.1002/(SICI)1097-4571(198903)40:2<115::AID-ASI6>3.0.CO;2-B]
	10	FAISST, S. Development of indexing functions based on probabilistic decision trees (in german). Diploma thesis, TH Darmstadt, FB Informatik, Datenverwaltungssysteme II, Darmstadt, Germany, 1990.
	11	D. H. Freeman, Jr., Applied categorical data analysis, Marcel Dekker, Inc., New York, NY, 1987
	12	FUHR, N. Probabilistisches Indexing und Retrieval. Dissertation, TH Darmstadt, Fachbereich Informatik, 1988. Available from Fachinformationszentrum Karlsruhe, Eggenstein- Leopoldshafen, Germany.
	13	Norbert Fuhr, Models for retrieval with probabilistic indexing, Information Processing and Management: an International Journal, v.25 n.1, p.55-72, 1989 [doi>10.1016/0306-4573(89)90091-5]
	14	Norbert Fuhr, Optimum polynomial retrieval functions based on the probability ranking principle, ACM Transactions on Information Systems (TOIS), v.7 n.3, p.183-204, July 1989 [doi>10.1145/65943.65944]
	15	M. Gordon, Probabilistic and genetic algorithms in document retrieval, Communications of the ACM, v.31 n.10, p.1208-1218, Oct. 1988 [doi>10.1145/63039.63044]
	16	KNORZ, G. Automatisches Indexieren als Erkennen Abstrakter Objekte. Niemeyer, Tiibingen, Germany, 1983.
	17	K. L. Kwok, An interpretation of index term weighting schemes based on document components, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.275-283, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253226]
	18	K. L. Kwok, A neural network for probabilistic information retrieval, Proceedings of the 12th annual international ACM SIGIR conference on Research and development in information retrieval, p.21-30, June 25-28, 1989, Cambridge, Massachusetts, United States
	19	K. L. Kwok , William Kuan, Experiments with document components for indexing and retrieval, Information Processing and Management: an International Journal, v.24 n.4, p.405-417, May 1988 [doi>10.1016/0306-4573(88)90044-1]
	20	M. E. Maron, Probabilistic approaches to the document retrieval problem, Proceedings of the 5th annual ACM conference on Research and development in information retrieval, p.98-107, May 18-20, 1982, West Berlin, Germany
	21	M. E. Maron , J. L. Kuhns, On Relevance, Probabilistic Indexing and Information Retrieval, Journal of the ACM (JACM), v.7 n.3, p.216-244, July 1960 [doi>10.1145/321033.321035]
	22	PFEIFER, U. R. Development of log-linear and linear-iterative indexing functions (in german). Diploma thesis, TH Darmstadt, FB Informatik, Datenverwaltungssysteme II, Darmstadt, Germany, 1990.
	23	QUINLAN, J.R. The effect of noise on concept learmng. In Machine Learning: An Artificial Intelligence Approach. Vol. /I, R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, Eds Morgan Kaufmann, Los Altos, Cahfornia, 1986, pp. 149-166.
	24	ROBERTSON, S. E., MARON, M. E., AND COOPER, W.S. Probabihty of relevance: A unification of two competing models for document retrieval. Inf Tech. Res. DRy. 1, (1982), 1-21.
	25	ROBERTSON, S. E., AND SPARCK JONES, K Relevance weighting of search terms. J. Am. Soc. Inf. Sci. 27, (1976), 129-146.
	26	S. E. Robertson , C. J. van Rijsbergen , M. F. Porter, Probabilistic models of indexing and searching, Proceedings of the 3rd annual ACM conference on Research and development in information retrieval, p.35-56, June 23-27, 1980, Cambridge, England
	27	Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0]
	28	SAL~CON, G, YANC, C S. AND Yu, C. T. A theory of term importance in automatic text analysis. J. Am. Soc. Inf. Sc~. 36, (1975), 33-44.
	29	T.M.T Sembok , C.J. van Rijsbergen, SILOL: a simple logical-linguistic document retrieval system, Information Processing and Management: an International Journal, v.26 n.1, p.111-134, 1990 [doi>10.1016/0306-4573(90)90012-Q]
	30	Alan F. Smeaton, Incorporating syntactic information into a document retrieval strategy: an investigation, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.103-113, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253193]
	31	TmTZE, A. Approximation of discrete probabihty distributions by dependence trees and their application as indexing functions (m german). Diploma thesm, TH Darmstadt, FB Informatik, Datenverwaltungssysteme II, Darmstadt, Germany, 1989
	32	Howard Robert Turtle, Inference networks for document retrieval, University of Massachusetts, Amherst, MA, 1991
	33	VAN RIJSBERGEN, C J A theoretical basis for the use of co-occurrence data in information retrieval. J. Doc. 33, (1977), 106-119.
	34	Andrew K. C. Wong , David K. Y. Chiu, Synthesizing statistical knowledge from incomplete mixed-mode data, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.9 n.6, p.796-805, November 1, 1987 [doi>10.1109/TPAMI.1987.4767986]
	35	S. K. M. Wong , Y. Y. Yao, A probability distribution model for information retrieval, Information Processing and Management: an International Journal, v.25 n.1, p.39-53, 1989 [doi>10.1016/0306-4573(89)90090-3]
	36	C. T. Yu , H. Mizuno, Two learning schemes in information retrieval, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.201-218, May 1988, Grenoble, France [doi>10.1145/62437.62451]
	37	C. T. Yu , G. Salton, Precision Weighting—An Effective Automatic Indexing Method, Journal of the ACM (JACM), v.23 n.1, p.76-88, Jan. 1976 [doi>10.1145/321921.321930]

CITED BY 45

	Norbert Fuhr , Thomas Rölleke, A probabilistic relational algebra for the integration of information retrieval and database systems, ACM Transactions on Information Systems (TOIS), v.15 n.1, p.32-66, Jan. 1997
	F. Crestani , C. J. van Rijsbergen, A study of probability kinematics in information retrieval, ACM Transactions on Information Systems (TOIS), v.16 n.3, p.225-255, July 1998
	Norbert Fuhr, Models in information retrieval, Lectures on information retrieval, Springer-Verlag New York, Inc., New York, NY, 2001
	William S. Cooper , Fredric C. Gey , Daniel P. Dabney, Probabilistic retrieval based on staged logistic regression, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.198-210, June 21-24, 1992, Copenhagen, Denmark
	Norbert Fuhr , Ulrich Pfeifer, Combining model-oriented and description-oriented approaches for probabilistic indexing, Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, p.46-56, October 13-16, 1991, Chicago, Illinois, United States
	Carlo Meghini , Fabrizio Sebastiani , Umberto Straccia, A model of multimedia information retrieval, Journal of the ACM (JACM), v.48 n.5, p.909-970, September 2001
	Fabio Crestani , Mounia Lalmas , Cornelis J. Van Rijsbergen , Iain Campbell, “Is this document relevant?…probably”: a survey of probabilistic models in information retrieval, ACM Computing Surveys (CSUR), v.30 n.4, p.528-552, Dec. 1998
	Norbert Fuhr, Integration of probabilistic fact and text retrieval, Proceedings of the 15th annual international ACM SIGIR conference on Research and development in information retrieval, p.211-222, June 21-24, 1992, Copenhagen, Denmark
	Robert W.P. Luk , H. V. Leong , Tharam S. Dillon , Alvin T.S. Chan , W. Bruce Croft , James Allan, A survey in indexing and searching XML documents, Journal of the American Society for Information Science and Technology, v.53 n.6, p.415-437, May, 2002
	Warren R. Greiff, A theory of term weighting based on exploratory data analysis, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.11-19, August 24-28, 1998, Melbourne, Australia
	Hsinchun Chen, Introduction to the JASIST special topic section on web retrieval and mining: a machine learning perspective, Journal of the American Society for Information Science and Technology, v.54 n.7, p.621-624, May 2003
	Fabrizio Sebastiani, Machine learning in automated text categorization, ACM Computing Surveys (CSUR), v.34 n.1, p.1-47, March 2002
	William S. Cooper, Some inconsistencies and misidentified modeling assumptions in probabilistic information retrieval, ACM Transactions on Information Systems (TOIS), v.13 n.1, p.100-111, Jan. 1995
	Norbert Gövert , Mounia Lalmas , Norbert Fuhr, A probabilistic description-oriented approach for categorizing web documents, Proceedings of the eighth international conference on Information and knowledge management, p.475-482, November 02-06, 1999, Kansas City, Missouri, United States
	Yiming Yang , Christopher G. Chute, An example-based mapping method for text categorization and retrieval, ACM Transactions on Information Systems (TOIS), v.12 n.3, p.252-277, July 1994
	Fredric C. Gey, Inferring probability of relevance using the method of logistic regression, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.222-231, July 03-06, 1994, Dublin, Ireland
	Peter Schäuble, SPIDER: a multiuser information retrieval system for semistructured and dynamic data, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.318-327, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States
	Brian T. Bartell , Garrison W. Cottrell , Richard K. Belew, Automatic combination of multiple ranked retrieval systems, Proceedings of the 17th annual international ACM SIGIR conference on Research and development in information retrieval, p.173-181, July 03-06, 1994, Dublin, Ireland
	A unified maximum likelihood approach to document retrieval, Journal of the American Society for Information Science and Technology, v.52 n.10, p.785-796, August 2001
	Wei Yi Liu , Ning Song, A fuzzy approach to classification of text documents, Journal of Computer Science and Technology, v.18 n.5, p.640-647, September 2003
	David Bodoff , Stephen Robertson, A new unified probabilistic model, Journal of the American Society for Information Science and Technology, v.55 n.6, p.471-487, April 2004
	Julia Hodges , Shiyun Yie , Sonal Kulkarni , Ray Reighart, Generation and Evaluation of Indexes for Chemistry Articles, Journal of Intelligent Information Systems, v.8 n.1, p.57-76, Jan./Feb. 1997
	Weiguo Fan , Ming Luo , Li Wang , Wensi Xi , Edward A. Fox, Tuning before feedback: combining ranking discovery and blind feedback for robust retrieval, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom
	Razvan Stefan Bot , Yi-fang Brook Wu, Improving document representations using relevance feedback: the RFA algorithm, Proceedings of the thirteenth ACM international conference on Information and knowledge management, November 08-13, 2004, Washington, D.C., USA
	Weiguo Fan , Michael D. Gordon , Praveen Pathak, A generic ranking function discovery framework by genetic programming for information retrieval, Information Processing and Management: an International Journal, v.40 n.4, p.587-602, May 2004
	David Bodoff, Relevance models to help estimate document and query parameters, ACM Transactions on Information Systems (TOIS), v.22 n.3, p.357-380, July 2004
	Bo-Yeong Kang , Sang-Jo Lee, Document indexing: a concept-based approach to term weight estimation, Information Processing and Management: an International Journal, v.41 n.5, p.1065-1080, September 2005
	Chris Buckley, The importance of proper weighting methods, Proceedings of the workshop on Human Language Technology, March 21-24, 1993, Princeton, New Jersey
	Bo-Yeong Kang , Dae-Won Kim , Sang-Jo Lee, Exploiting concept clusters for content-based information retrieval, Information Sciences—Informatics and Computer Science: An International Journal, v.170 n.2-4, p.443-462, 25 February 2005
	Sandor Dominich, Connectionist interaction information retrieval, Information Processing and Management: an International Journal, v.39 n.2, p.167-193, March 2003
	ChengXiang Zhai , John Lafferty, A risk minimization framework for information retrieval, Information Processing and Management: an International Journal, v.42 n.1, p.31-55, January 2006
	David Bodoff , Samuel Po-Shing Wong, Documents and queries as random variables: History and implications: Research Articles, Journal of the American Society for Information Science and Technology, v.57 n.9, p.1138-1154, July 2006
	Norbert Fuhr , Ulrich Pfeifer, Probabilistic information retrieval as a combination of abstraction, inductive learning, and probabilistic assumptions, ACM Transactions on Information Systems (TOIS), v.12 n.1, p.92-115, Jan. 1994
	A. Georgakis , H. Li, User behavior modeling and content based speculative web page prefetching, Data & Knowledge Engineering, v.59 n.3, p.770-788, December 2006
	Hongyuan Zha , Zhaohui Zheng , Haoying Fu , Gordon Sun, Incorporating query difference for learning retrieval functions in world wide web search, Proceedings of the 15th ACM international conference on Information and knowledge management, November 06-11, 2006, Arlington, Virginia, USA
	Dmitri Roussinov , Weiguo Fan, Discretization based learning approach to information retrieval, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.153-160, October 06-08, 2005, Vancouver, British Columbia, Canada
	Dmitri Roussinov , Weiguo Fan, Discretization based learning approach to information retrieval, Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, p.153-160, October 06-08, 2005, Vancouver, British Columbia, Canada
	Shihao Ji , Ke Zhou , Ciya Liao , Zhaohui Zheng , Gui-Rong Xue , Olivier Chapelle , Gordon Sun , Hongyuan Zha, Global ranking by exploiting user clicks, Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, July 19-23, 2009, Boston, MA, USA
	Weiguo Fan , Michael D. Gordon , Praveen Pathak, Genetic Programming-Based Discovery of Ranking Functions for Effective Web Search, Journal of Management Information Systems, v.21 n.4, p.37-56, Number 4/Spring 2005
	Julia Hodges , Shiyun Yie , Ray Reighart , Lois Boggess, An automated system that assists in the generation of document indexes, Natural Language Engineering, v.2 n.2, p.137-160, June 1996
	Henrik Nottelmann , Norbert Fuhr, From Retrieval Status Values to Probabilities of Relevance for Advanced IR Applications, Information Retrieval, v.6 n.3-4, p.363-388, September-December 2003
	Ian Ruthven , Mounia Lalmas, A survey on the use of relevance feedback for information access systems, The Knowledge Engineering Review, v.18 n.2, p.95-145, June 2003
	Quan Wang , Yiu-Kai Ng, An Ontology-Based Binary-Categorization Approach for Recognizing Multiple-Record Web Documents Using a Probabilistic Retrieval Model, Information Retrieval, v.6 n.3-4, p.295-332, September-December 2003
	Yu-Chuan Chang , Shyi-Ming Chen , Churn-Jung Liau, Multilabel text categorization based on a new linear classifier learning method and a category-sensitive refinement method, Expert Systems with Applications: An International Journal, v.34 n.3, p.1948-1953, April, 2008
	J. Villaverde , A. Persson , D. Godoy , A. Amandi, Supporting the discovery and labeling of non-taxonomic relationships in ontology learning, Expert Systems with Applications: An International Journal, v.36 n.7, p.10288-10294, September, 2009

INDEX TERMS

Primary Classification:
H. Information Systems
H.3 INFORMATION STORAGE AND RETRIEVAL
H.3.1 Content Analysis and Indexing
Subjects: Indexing methods

Additional Classification:
G. Mathematics of Computing
G.1 NUMERICAL ANALYSIS
G.1.2 Approximation
Subjects: Least squares approximation

H. Information Systems
H.3 INFORMATION STORAGE AND RETRIEVAL
H.3.3 Information Search and Retrieval
Subjects: Retrieval models

I. Computing Methodologies
I.2 ARTIFICIAL INTELLIGENCE
I.2.6 Learning
Subjects: Parameter learning

General Terms:
Algorithms, Design, Experimentation, Theory

Keywords:
complex document representation, linear indexing functions, linear retrieval functions, probabilistic indexing, probabilistic retrieval, relevance descriptions

REVIEWS

"Karen Sparck-Jones : Reviewer"

The authors describe a sophisticated approach to probabilistic document indexing and retrieval, and give the results of tests with it. The main problem in using past searches to predict the relevance of a specific document to a query, given th more...

"Ian Hugh Witten : Reviewer","Halim Zahran : Reviewer","Razak Zadah : Reviewer"

A probabilistic approach to the problem of assigning weights to index terms in documents is described. The result of processing a set of queries is viewed as a space of query-document pairs with attached relevance judgments (relevant or not re more...

Collaborative Colleagues:

Norbert Fuhr: colleagues

Chris Buckley: colleagues

Adobe Acrobat

QuickTime

Windows Media Player

Real Player