Model-directed Web transactions under constrained modalities

ABSTRACT

Online transactions (e.g., buying a book on the Web) typically involve a number of steps spanning several pages. Conducting such transactions under constrained interaction modalities as exemplified by small screen handhelds or interactive speech interfaces—the primary mode of communication for visually impaired individuals—is a strenuous, fatigue-inducing activity. But usually one needs to browse only a small fragment of a Web page to perform a transactional step such as a form fillout, selecting an item from a search results list, and so on. We exploit this observation to develop an automata-based process model that delivers only the “relevant” page fragments at each transactional step, thereby reducing information overload on such narrow interaction bandwidths. We realize this model by coupling techniques from content analysis of Web documents, automata learning and statistical classification. The process model and associated techniques have been incorporated into Guide-O, a prototype system that facilitates online transactions using speech/keyboard interface (Guide-O-Speech), or with limited-display size handhelds (Guide-O-Mobile). Performance of Guide-O and its user experience are reported.

REFERENCES

• 1: Agarwal, S., Handschuh, S., and Staab, S. 2003. Surfing the service web. In International Semantic Web Conference (ISWC). 211--216.
• 2: Angluin, D. 1978. On the complexity of minimum inference of regular sets. Inform. Control 39, 3, 337--350.
• 3: Chieko Asakawa , Takashi Itoh, User interface of a Home Page Reader, Proceedings of the third international ACM conference on Assistive technologies, p.149-156, April 15-17, 1998, Marina del Rey, California, United States [doi>10.1145/274497.274526]
• 4: Shumeet Baluja, Browsing on small screens: recasting web-page segmentation into an efficient machine learning framework, Proceedings of the 15th international conference on World Wide Web, May 23-26, 2006, Edinburgh, Scotland [doi>10.1145/1135777.1135788]
• 5: Orkut Buyukkokten , Hector Garcia-Molina , Andreas Paepcke, Seeing the whole in parts: text summarization for web browsing on handheld devices, Proceedings of the 10th international conference on World Wide Web, p.652-662, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372178]
• 6: Orkut Buyukkokten , Hector Garcia-Molina , Andreas Paepcke , Terry Winograd, Power browser: efficient Web browsing for PDAs, Proceedings of the SIGCHI conference on Human factors in computing systems, p.430-437, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332470]
• 7: Chen, L., Shadbolt, N., Goble, C. A., Tao, F., Cox, S. J., Puleston, C., and Smart, P. R. 2003. Towards a knowledge-based approach to semantic service composition. In Proceedings of the International Semantic Web Conference (ISWC). 319--334.
• 8: Yu Chen , Wei-Ying Ma , Hong-Jiang Zhang, Detecting web page structure for adaptive viewing on small form factor devices, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775184]
• 9: Ed H. Chi , Peter Pirolli , Kim Chen , James Pitkow, Using information scent to model user information needs and actions and the Web, Proceedings of the SIGCHI conference on Human factors in computing systems, p.490-497, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365325]
• 10: Philipp Cimiano , Siegfried Handschuh , Steffen Staab, Towards the self-annotating web, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988735]
• 11: Stephen Dill , Nadav Eiron , David Gibson , Daniel Gruhl , R. Guha , Anant Jhingran , Tapas Kanungo , Sridhar Rajagopalan , Andrew Tomkins , John A. Tomlin , Jason Y. Zien, SemTag and seeker: bootstrapping the semantic web via automated semantic annotation, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775178]
• 12: Murielle Florins , Jean Vanderdonckt, Graceful degradation of user interfaces as a design method for multiplatform systems, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964469]
• 13: Gold, E. M. 1978. Complexity of automaton identification from given data. Inform. Control 37, 3, 302--320.
• 14: Dan Gusfield, Algorithms on strings, trees, and sequences: computer science and computational biology, Cambridge University Press, New York, NY, 1997
• 15: Hammond, B., Sheth, A., and Kochut, K. 2002. Semantic enhancement engine: A modular document enhancement platform for semantic applications over heterogenous content. In Real World Semantic Applications, V. Kashyap and L. Shklar, Eds. IOS Press.
• 16: Simon Harper , Neha Patel, Gist summaries for visually impaired surfers, Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, October 09-12, 2005, Baltimore, MD, USA [doi>10.1145/1090785.1090804]
• 17: Heflin, J., Hendler, J. A., and Luke, S. 2003. SHOE: A blueprint for the semantic web. In Spinning the Semantic Web, D. Fensel, J. A. Hendler, H. Lieberman, and W. Wahlster, Eds. MIT Press, 29--63.
• 18: Hess, A., Johnston, E., and Kushmerick, N. 2004. Assam: A tool for semi-automatically annotating semantic web services. In Proceedings of the International Semantic Web Conference (ISWC). 320--334.
• 19: Anita W. Huang , Neel Sundaresan, A semantic transcoding system to adapt Web services for users with disabilities, Proceedings of the fourth international ACM conference on Assistive technologies, p.156-163, November 13-15, 2000, Arlington, Virginia, United States [doi>10.1145/354324.354363]
• 20: IPhone. http://www.apple.com/iphone/.
• 21: JAWS Screen Reader. http://www.freedomscientific.com.
• 22: Kaasinen, E., Aaltonen, M., Kolari, J., Melakoski, S., and Laakko, T. 2000. Two approaches to bringing internet services to wap devices. In Proceedings of the International World Wide Web Conference (WWW).
• 23: José Kahan , Marja-Ritta Koivunen, Annotea: an open RDF infrastructure for shared Web annotations, Proceedings of the 10th international conference on World Wide Web, p.623-632, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372166]
• 24: Alberto H. F. Laender , Berthier A. Ribeiro-Neto , Altigran S. da Silva , Juliana S. Teixeira, A brief survey of web data extraction tools, ACM SIGMOD Record, v.31 n.2, June 2002 [doi>10.1145/565117.565137]
• 25: Milic-Frayling, N. and Sommerer, R. 2002. Smartview: Flexible viewing of web page contents. In Proceedings of the International World Wide Web Conference (WWW). ACM Press, 172.
• 26: Saikat Mukherjee , I. V. Ramakrishnan , Amarjeet Singh, Bootstrapping Semantic Annotation for Content-Rich HTML Documents, Proceedings of the 21st International Conference on Data Engineering, p.583-593, April 05-08, 2005 [doi>10.1109/ICDE.2005.28]
• 27: Murphy, K. 1996. Passively learning finite automata. Tech. rep., Janta Fe Institute.
• 28: NetECHO{tm}. http://www.internetspeech.com.
• 29: Oncina, J. and Garc, P. 1991. Inferring Regular Languages in Polynomial Update Time. World Scientific Publishing.
• 30: Abhijit A. Patil , Swapna A. Oundhakar , Amit P. Sheth , Kunal Verma, Meteor-s web service annotation framework, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988747]
• 31: E. Pontelli , D. Gillan , W. Xiong , E. Saad , G. Gupta , A. I. Karshmer, Navigation of HTML tables, frames, and XML fragments, Proceedings of the fifth international ACM conference on Assistive technologies, July 08-10, 2002, Edinburgh, Scotland [doi>10.1145/638249.638256]
• 32: Popov, B., Kiryakov, A., Kirilov, A., Manov, D., Ognyanoff, D., and Goranov, M. 2003. Kim---semantic annotation platform. In Proceedings of the International Semantic Web Conference (ISWC).
• 33: I. V. Ramakrishnan , Amanda Stent , Guizhen Yang, Hearsay: enabling audio browsing on hypertext content, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988684]
• 34: T. V. Raman , B. Rovan, Audio System for Technical Readings, Springer-Verlag New York, Inc., Secaucus, NJ, 1999
• 35: Lakshmish Ramaswamy , Arun Iyengar , Ling Liu , Fred Douglis, Automatic detection of fragments in dynamically generated web pages, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988732]
• 36: Jinghai Rao , Peep Küngas , Mihhail Matskin, Logic-based Web Services Composition: From Service Description to Process Model, Proceedings of the IEEE International Conference on Web Services, p.446, June 06-09, 2004 [doi>10.1109/ICWS.2004.71]
• 37: John T. Richards , Vicki L. Hanson, Web accessibility: a broader view, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988683]
• 38: Marta Sabou , Chris Wroe , Carole Goble , Gilad Mishne, Learning domain ontologies for Web service descriptions: an experiment in bioinformatics, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1060745.1060776]
• 39: Ruihua Song , Haifeng Liu , Ji-Rong Wen , Wei-Ying Ma, Learning block importance models for web pages, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988700]
• 40: Hironobu Takagi , Chieko Asakawa , Kentarou Fukuda , Junji Maeda, Site-wide annotation: reconstructing existing pages to be accessible, Proceedings of the fifth international ACM conference on Assistive technologies, July 08-10, 2002, Edinburgh, Scotland [doi>10.1145/638249.638265]
• 41: Thunderhawk. http://www.bitstream.com/wireless/index.html.
• 42: Traverso, P. and Pistore, M. 2004. Automated composition of semantic web services into executable processes. In Proceedings of the International Semantic Web Conference (ISWC).
• 43: W. M. P. van der Aalst , A. J. M. M. Weijters, Process mining: a research agenda, Computers in Industry, v.53 n.3, p.231-244, April 2004 [doi>10.1016/j.compind.2003.10.001]
• 44: Andreas Wombacher , Peter Fankhauser , Erich Neuhold, Transforming BPEL into Annotated Deterministic Finite State Automata for Service Discovery, Proceedings of the IEEE International Conference on Web Services, p.316, June 06-09, 2004 [doi>10.1109/ICWS.2004.117]
• 45: Xiangye Xiao , Qiong Luo , Dan Hong , Hongbo Fu, Slicing*-tree based web page transformation for small displays, Proceedings of the 14th ACM international conference on Information and knowledge management, October 31-November 05, 2005, Bremen, Germany [doi>10.1145/1099554.1099638]
• 46: Christopher C. Yang , Fu Lee Wang, Fractal summarization for mobile devices to access large documents on the web, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775183]
• 47: Lan Yi , Bing Liu , Xiaoli Li, Eliminating noisy information in Web pages for data mining, Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining, August 24-27, 2003, Washington, D.C. [doi>10.1145/956750.956785]
• 48: Xinyi Yin , Wee Sun Lee, Using link analysis to improve layout on mobile devices, Proceedings of the 13th international conference on World Wide Web, May 17-20, 2004, New York, NY, USA [doi>10.1145/988672.988718]
• 49: Shipeng Yu , Deng Cai , Ji-Rong Wen , Wei-Ying Ma, Improving pseudo-relevance feedback in web information retrieval using web page segmentation, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775155]
• 50: Zajicek, M., Powell, C., and Reeves, C. 1999. Web search and orientation with brookestalk. In Proceedings of Technology and Persons with Disabilities Conference.
• 51: Zhen Zhang , Bin He , Kevin Chen-Chuan Chang, Understanding Web query interfaces: best-effort parsing with hidden syntax, Proceedings of the 2004 ACM SIGMOD international conference on Management of data, June 13-18, 2004, Paris, France [doi>10.1145/1007568.1007583]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.5 Document Capture
          Subjects: Document analysis

General Terms:
Algorithms, Human Factors

Keywords:
Web transaction, assistive device, content adaption