Natural language processing and e-Government: crime information extraction from heterogeneous data sources

ABSTRACT

Much information that could help solve and prevent crimes is never gathered because the reporting methods available to citizens and law enforcement personnel are not optimal. Detectives do not have sufficient time to interview crime victims and witnesses. Moreover, many victims and witnesses are too scared or embarrassed to report incidents. We are developing an interviewing system that will help collect such information. We report here on one component, the crime information extraction module, which uses natural language processing to extract crime information from police reports, newspaper articles, and victims' and witnesses' crime narratives. We tested our approach with two types of document: police and witness narrative reports. Our algorithms extract crime-related information, namely weapons, vehicles, time, people, clothes, and locations. We achieved high precision (96%) and recall (83%) for police narrative reports and comparable precision (93%) but somewhat lower recall (77%) for witness narrative reports. The difference in recall was significant at p < .05. We then used a spell checker to evaluate if this would help with witness narrative processing. We found that both precision (94%) and recall (79%) improved slightly.

REFERENCES

• 1: E. D. d. Leeuw, "Reducing Missing Data in Surveys: An Overview of Methods," Quality and Quantity, vol. 35, pp. 147--160, 2001.
• 2: Michael Chau , Jennifer J. Xu , Hsinchun Chen, Extracting meaningful entities from police narrative reports, Proceedings of the 2002 annual national conference on Digital government research, p.1-5, May 19-22, 2002, Los Angeles, California
• 3: A. Iriberri and G. Leroy, "Natural Language Processing and e-Government: Extracting Reusable Crime Report Information," in Information Reuse and Integration, 2007. IRI 2007. IEEE International Conference on, Las Vegas, NV, USA, 2007, pp. 221--226.
• 4: L. Hirschman, A. Yeh, C. Blaschke, and A. Valencia, "Overview of BioCreAtIvE: Critical Assessment of Information Extraction for Biology," BMC Bioinformatics, vol. 6, May 2005.
• 5: Hsinchun Chen , Wingyan Chung , Jennifer Jie Xu , Gang Wang , Yi Qin , Michael Chau, Crime Data Mining: A General Framework and Some Examples, Computer, v.37 n.4, p.50-56, April 2004 [doi>10.1109/MC.2004.1297301]
• 6: A. M. Lyons, J. Michael W. Packer, M. B. Thomason, J. C. Wesley, P. J. Hansen, J. H. Conklin, and D. E. Brown, "Uniform Crime Report "SuperClean" Data Cleaning Tool," Systems and Information Engineering Design Symposium, 2006 IEEE, pp. 14--18, 2006.
• 7: Ronen Feldman , Benjamin Rosenfeld , Moshe Fresko, TEG—a hybrid approach to information extraction, Knowledge and Information Systems, v.9 n.1, p.1-18, January 2006 [doi>10.1007/s10115-005-0204-y]
• 8: D. Maynard, K. Bontcheva, and H. Cunningham, "Towards a Semantic Extraction of Named Entities," in Recent Advances in Natural Language Processing Bulgaria, 2003.
• 9: K. Pastra, D. Maynard, O. Hamza, H. Cunningham, and YorickWilks, "How Feasible is the Reuse of Grammars for Named Entity Recognition?," in In Proceedings of the Language Resources and Evaluation Conference, 2002, pp. 1412--1418.
• 10: K. Bontcheva, M. Dimitrov, D. Maynard, V. Tablan, and H. Cunningham, "Shallow Methods for Named Entity Coreference Resolution," in Workshop TALN 2002 Nancy, France, 2002.
• 11: Ronen Feldman, Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data, Cambridge University Press, New York, NY, 2006
• 12: Shyam Varan Nath, Crime Pattern Detection Using Data Mining, Proceedings of the 2006 IEEE/WIC/ACM international conference on Web Intelligence and Intelligent Agent Technology, p.41-44, December 18-22, 2006 [doi>10.1109/WI-IATW.2006.55]
• 13: B. T. Pentland, "Building Process Theory with Narrative: From Description to Explanation," Academy of Management Review, vol. 24, pp. 711--724, Oct. 1999.
• 14: A. Borthwick, J. Sterling, E. Agichtein, and R. Grishman, "NYU: Description of the MENE Named Entity System as Used in MUC-7," in in Proceedings of the Seventh Message Understanding Conference (MUC-7), April 1998.
• 15: Mstislav Maslennikov , Hai-Kiat Goh , Tat-Seng Chua, ARE: instance splitting strategies for dependency relation-based information extraction, Proceedings of the COLING/ACL on Main conference poster sessions, p.571-578, July 17-18, 2006, Sydney, Australia
• 16: Brian Roark , Eugene Charniak, Noun-phrase co-occurrence statistics for semiautomatic semantic lexicon construction, Proceedings of the 17th international conference on Computational linguistics, p.1110-1116, August 10-14, 1998, Montreal, Quebec, Canada
• 17: Dayne Freitag, Machine Learning for Information Extraction in Informal Domains, Machine Learning, v.39 n.2-3, p.169-202, May-June 2000 [doi>10.1023/A:1007601113994]
• 18: Y. Liu, Y. Lin, and Z. Chen, "Using Hidden Markov Model for Information Extraction Based on Multiple Templates," in Natural Language Processing and Knowledge Engineering, 2003. Proceedings, 2003, pp. 394--399.
• 19: D. Maynard, M. Yankova, A. Kourakis, and A. Kokossis, Ontology-Based Information Extraction For Market Monitoring And Technology Watch. Heraklion, Crete, 2005.
• 20: H. Cunningham, "GATE, a General Architecture for Text Engineering," Computers and the Humanities, vol. 36, pp. 223--254, May 2002.
• 21: Diana Maynard , Kalina Bontcheva , Horacio Saggion , Hamish Cunningham , Oana Hamza, Using a text engineering framework to build an extendable and portable IE-based summarisation system, Proceedings of the ACL-02 Workshop on Automatic Summarization, p.19-26, July 11-12, 2002, Phildadelphia, Pennsylvania [doi>10.3115/1118162.1118165]
• 22: Mark Hepple, Independence and commitment: assumptions for rapid training and execution of rule-based POS taggers, Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, p.278-277, October 03-06, 2000, Hong Kong [doi>10.3115/1075218.1075254]
• 23: L. A. Ramshaw and M. P. Marcus, "Text Chunking using Transformation-Based Learning," in Proceedings of the Third Workshop on Very Large Corpora, 1995, pp. 82--94.
• 24: A. Memon and R. Bull, "The Cognitive Interview - Its Origins, Empirical Support, Evaluation and Practical Implications," Journal Of Community & Applied Social Psychology, vol. 1, pp. 291--307, 1991.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.0 General

General Terms:
Algorithms, Design, Measurement, Performance

Keywords:
information extraction, natural language processing