Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

This paper addresses the problem of making text mining results more comprehensible to humanities scholars, journalists, intelligence analysts, and other researchers, in order to support the analysis of text collections. Our system, FeatureLens¹, visualizes a text collection at several levels of granularity and enables users to explore interesting text patterns. The current implementation focuses on frequent itemsets of n-grams, as they capture the repetition of exact or similar expressions in the collection. Users can find meaningful co-occurrences of text patterns by visualizing them within and across documents in the collection. This also permits users to identify the temporal evolution of usage such as increasing, decreasing or sudden appearance of text patterns. The interface could be used to explore other text features as well. Initial studies suggest that FeatureLens helped a literary scholar and 8 users generate new hypotheses and interesting insights using 2 text collections.

REFERENCES

• 1: Rakesh Agrawal , Ramakrishnan Srikant, Fast Algorithms for Mining Association Rules in Large Databases, Proceedings of the 20th International Conference on Very Large Data Bases, p.487-499, September 12-15, 1994
• 2: Church, K. W., and Helfman, J. I., Dotplot: A Program for Exploring Self-Similarity in Millions of Lines of Text and Code, In Proc. of the 24th Symposium on the Interface, Computing Science and Statistics V24, 58--67. 1992.
• 3: Stephen G. Eick , Joseph L. Steffen , Eric E. Sumner, Jr., Seesoft-A Tool for Visualizing Line Oriented Software Statistics, IEEE Transactions on Software Engineering, v.18 n.11, p.957-968, November 1992 [doi>10.1109/32.177365]
• 4: Jean-Daniel Fekete , Nicole Dufournaud, Compus: visualization and analysis of structured documents for understanding social life in the 16th century, Proceedings of the fifth ACM conference on Digital libraries, p.47-55, June 02-07, 2000, San Antonio, Texas, United States [doi>10.1145/336597.336632]
• 5: Frank, A. C., Amiri, H., Andersson, S., Genome Deterioration: loss of repeated sequences and accumulation of junk DNA. Genetica, Vol. 115, No. 1, 1--12. 2002.
• 6: Kurtz, S & Schleiermacher, C. REPuter: fast computation of maximal repeats in complete genomes. Bioinformatics 15, 426--427. 1999.
• 7: Gerard Lommerse , Freek Nossin , Lucian Voinea , Alexandru Telea, The Visual Code Navigator: An Interactive Toolset for Source Code Investigation, Proceedings of the Proceedings of the 2005 IEEE Symposium on Information Visualization, p.4, October 23-25, 2005 [doi>10.1109/INFOVIS.2005.33]
• 8: NY Times: The State of the Union in Words. http://www.nytimes.com/ref/washington/20070123_STATEOFUNION.html
• 9: Paley, W. B. TextArc: Showing Word Frequency and Distribution in Text. Poster presented at IEEE Symposium on Information Visualization. 2002.
• 10: J. Pei and J. Han and R. Mao, CLOSET: An Efficient Algorithm for Mining Frequent Closed Itemsets, ACM SIGMOD, Workshop on Research Issues in Data Mining and Knowledge Discovery, 21--30. 2000.
• 11: Catherine Plaisant , James Rose , Bei Yu , Loretta Auvil , Matthew G. Kirschenbaum , Martha Nell Smith , Tanya Clement , Greg Lord, Exploring erotics in Emily Dickinson's correspondence with text mining and visual interfaces, Proceedings of the 6th ACM/IEEE-CS joint conference on Digital libraries, June 11-15, 2006, Chapel Hill, NC, USA [doi>10.1145/1141753.1141781]
• 12: Data to Knowledge (D2K) and Text to knowledge (T2K), NCSA. http://alg.ncsa.uiuc.edu/do/tools.
• 13: Thomas, J. J. and Cook, K. A. (eds.), Illuminating the Path: Research and Development Agenda for Visual Analytics, IEEE. 2005.
• 14: Aravindan Veerasamy , Nicholas J. Belkin, Evaluation of a tool for visualization of information retrieval results, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.85-92, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243218]
• 15: Martin Wattenberg, Arc Diagrams: Visualizing Structure in Strings, Proceedings of the IEEE Symposium on Information Visualization (InfoVis'02), p.110, October 28-29, 2002
• 16: J. A. Wise , J. J. Thomas , K. Pennock , D. Lantrip , M. Pottier , A. Schur , V. Crow, Visualizing the non-visual: spatial analysis and interaction with information from text documents, Proceedings of the 1995 IEEE Symposium on Information Visualization, p.51, October 30-31, 1995, Atlanta, Georgia

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

General Terms:
Algorithms, Design, Experimentation, Human Factors, Measurement

Keywords:
digital humanities, frequent closed itemsets, n-grams, text mining, user interface