Crosslingual location search

ABSTRACT

Address geocoding, the process of finding the map location for a structured postal address, is a relatively well-studied problem. In this paper we consider the more general problem of crosslingual location search, where the queries are not limited to postal addresses, and the language and script used in the search query is different from the one in which the underlying data is stored. To the best of our knowledge, our system is the first crosslingual location search system that is able to geocode complex addresses. We use a statistical machine transliteration system to convert location names from the script of the query to that of the stored data. However, we show that it is not sufficient to simply feed the resulting transliterations into a monolingual geocoding system, as the ambiguity inherent in the conversion drastically expands the location search space and significantly lowers the quality of results. The strength of our approach lies in its integrated, end-to-end nature: we use abstraction and fuzzy search (in the text domain) to achieve maximum coverage despite transliteration ambiguities, while applying spatial constraints (in the geographic domain) to focus only on viable interpretations of the query. Our experiments with structured and unstructured queries in a set of diverse languages and scripts (Arabic, English, Hindi and Japanese) searching for locations in different regions of the world, show full crosslingual location search accuracy at levels comparable to that of commercial monolingual systems. We achieve these levels of performance using techniques that may be applied to crosslingual searches in any language/script, and over arbitrary spatial data.

REFERENCES

• 1: Yaser Al-Onaizan , Kevin Knight, Machine transliteration of names in Arabic text, Proceedings of the ACL-02 workshop on Computational approaches to semitic languages, p.1-13, July 11, 2002, Philadelphia, Pennsylvania [doi>10.3115/1118637.1118642]
• 2: Surajit Chaudhuri , Kris Ganjam , Venkatesh Ganti , Rajeev Motwani, Robust and efficient fuzzy match for online data cleaning, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872796]
• 3: Christen, P., Churches, T. and Willmore, A. A probabilistic geocoding system based on a national address file. In Proc. 3rd Australasian Data Mining Conf., 2004.
• 4: CLEF Forum. http://www.clef-campaign.org/.
• 5: Irene Gargantini, An effective way to represent quadtrees, Communications of the ACM, v.25 n.12, p.905-910, Dec 1982 [doi>10.1145/358728.358741]
• 6: GeoCLEF. http://ir.shef.ac.uk/geoclef/.
• 7: Goldberg, D. W., Wilson, J. P., and Knoblock, C. A. From text to geographic coordindates: The current state of geocoding. In J. Urban and Regional Information Systems Assoc., 2006.
• 8: Joshi, T., Joy, J., and Sengar, V. Robust Location Search. Technical Report MSR-TR-2008-41, Microsoft Research, 2008.
• 9: Goto, I., Kato, N., Uratani, N. and Ehara, T. Transliteration considering context information based on the Maximum entropy method. In Proc. MT Summit IX, 2004.
• 10: Kevin Knight , Jonathan Graehl, Machine transliteration, Computational Linguistics, v.24 n.4, p.599-612, December 1998
• 11: A. Kumaran , Tobias Kellner, A generic framework for machine transliteration, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277876]
• 12: Lin, Dekang. MaxEnt Classifier. 2003. http://www.cs.ualberta.ca/~lindek/maxent.tgz.
• 13: Oh,J., Choi, K. & Isahara, H. A comparison of different machine transliteration models. Artificial Intelligence Research, 2006.
• 14: Bruno Pouliquen , Ralf Steinberger , Camelia Ignat , Tom De Groeve, Geographical information recognition and visualization in texts written in various languages, Proceedings of the 2004 ACM symposium on Applied computing, March 14-17, 2004, Nicosia, Cyprus [doi>10.1145/967900.968115]
• 15: Graham Rhind, Global SourceBook of Address Data Management: A Guide to Address Formats and Data in 194 Countries, Gower Publishing Co., Brookfield, VT 05036, 1998
• 16: Russell, R. Soundex. US Patent 1,261,167, 1918.
• 17: Vibhuti Sengar , Tanuja Joshi , Joseph Joy , Samarth Prakash , Kentaro Toyama, Robust location search from text queries, Proceedings of the 15th annual ACM international symposium on Advances in geographic information systems, November 07-09, 2007, Seattle, Washington [doi>10.1145/1341012.1341044]
• 18: Paul Viola , Mukund Narasimhan, Learning to extract information from semi-structured text using a discriminative context free grammar, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076091]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

General Terms:
Algorithms, Design

Keywords:
address geocoding, crosslingual information retrieval, location search