HyperReal: a hypermedia model for mixed reality

ABSTRACT

This paper describes a generic hypermedia model that is used as a framework for building context aware and mixed reality applications. It can handle different media elements, and it defines a presentation scheme that abstracts several relevant navigation concepts, including link awareness. The model specifies a base structure for the relation between spaces, either real or virtual, and supports contextual mechanisms. Additionally, it establishes a way to correlate real/virtual world objects with information present in the hypermedia graph. It also includes store/replay mechanisms that can be used to repurpose the content in new ways, including storytelling applications. The proposed model is being tested in a gaming and storytelling environment that integrates the real world, media elements and virtual 3D worlds. The paper presents the overall framework, the current implementation and evaluates its usage in the prototype application.

REFERENCES

• 1: R. Azuma, "A Survey of Augmented Reality", Presence, vol. 6, no. 4, pp. 355--285, 1997.
• 2: Ronald Azuma , Yohan Baillot , Reinhold Behringer , Steven Feiner , Simon Julier , Blair MacIntyre, Recent Advances in Augmented Reality, IEEE Computer Graphics and Applications, v.21 n.6, p.34-47, November 2001 [doi>10.1109/38.963459]
• 3: Jörg Baus , Antonio Krüger , Wolfgang Wahlster, A resource-adaptive mobile navigation system, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502723]
• 4: Francesco Bellotti , Riccardo Berta , Alessandro De Gloria , Massimiliano Margarone, User Testing a Hypermedia Tour Guide, IEEE Pervasive Computing, v.1 n.2, p.33-41, April 2002 [doi>10.1109/MPRV.2002.1012335]
• 5: Mark Billinghurst , Hirkazu Kato , Ivan Poupyrev, The MagicBook—Moving Seamlessly between Reality and Virtuality, IEEE Computer Graphics and Applications, v.21 n.3, p.6-8, May 2001
• 6: S. Bjork, J. Falk, R. Hansson, P. Ljungstrand, "Pirates! Using the Physical World as a Game Board". Interact 2001.
• 7: Peter Brusilovsky, Adaptive Hypermedia, User Modeling and User-Adapted Interaction, v.11 n.1-2, p.87-110, 2001 [doi>10.1023/A:1011143116306]
• 8: Nolan Bushnell, Relationships between fun and the computer business, Communications of the ACM, v.39 n.8, p.31-37, Aug. 1996 [doi>10.1145/232014.232025]
• 9: Brad Campbell , Joseph M. Goodman, HAM: a general-purpose hypertext abstract machine, Proceeding of the ACM conference on Hypertext, p.21-32, November 1987, Chapel Hill, North Carolina, United States [doi>10.1145/317426.317429]
• 10: Teresa Chambel , Nuno Guimarães, Context perception in video-based hypermedia spaces, Proceedings of the thirteenth ACM conference on Hypertext and hypermedia, June 11-15, 2002, College Park, Maryland, USA [doi>10.1145/513338.513365]
• 11: Keith Cheverst , Keith Mitchell , Nigel Davies, The role of adaptive hypermedia in a context-aware tourist GUIDE, Communications of the ACM, v.45 n.5, May 2002 [doi>10.1145/506218.506244]
• 12: Paul De Bra , Geert-Jan Houben , Hongjing Wu, AHAM: a Dexter-based reference model for adaptive hypermedia, Proceedings of the tenth ACM Conference on Hypertext and hypermedia : returning to our diverse roots: returning to our diverse roots, p.147-156, February 21-25, 1999, Darmstadt, Germany [doi>10.1145/294469.294508]
• 13: Paul De Bra , Geert-Jan Houben , Yoram Kornatzky, An extensible data model for hyperdocuments, Proceedings of the ACM conference on Hypertext, p.222-231, November 30-December 04, 1992, Milan, Italy [doi>10.1145/168466.168530]
• 14: Sastry Duri , Alan Cole , Jonathan Munson , Jim Christensen, An approach to providing a seamless end-user experience for location-aware applications, Proceedings of the 1st international workshop on Mobile commerce, p.20-25, July 2001, Rome, Italy [doi>10.1145/381461.381465]
• 15: M. Flintham, R. Anastasi, S. Benford, T. Hemmings, A. Crabtree, C. Greenhalgh, T. Rodden, N. Tandavanitj, M. Adams, J. Row-Farr, "Where On-Line Meets On-The-Streets: Experiences With Mobile Mixed Reality Games", Technical Report Equator-02-008, Equator, September 2002.
• 16: Kaj Grønbæk , Randall H. Trigg, Design issues for a Dexter-based hypermedia system, Proceedings of the ACM conference on Hypertext, p.191-200, November 30-December 04, 1992, Milan, Italy [doi>10.1145/168466.168525]
• 17: K. Grönbæk, P. P. Vestergaard, P. Orbaek, "Towards Geo-Spatial Hypermedia: Concepts and Prototype Implementation", ACM Hypertext 2002, pp. 117--126.
• 18: Frank G. Halasz , Thomas P. Moran , Randall H. Trigg, Notecards in a nutshell, Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, p.45-52, April 05-09, 1987, Toronto, Ontario, Canada
• 19: Frank Halasz , Mayer Schwartz, The Dexter hypertext reference model, Communications of the ACM, v.37 n.2, p.30-39, Feb. 1994 [doi>10.1145/175235.175237]
• 20: Lynda Hardman , Dick C. A. Bulterman , Guido van Rossum, The Amsterdam hypermedia model: adding time and context to the Dexter model, Communications of the ACM, v.37 n.2, p.50-62, Feb. 1994 [doi>10.1145/175235.175239]
• 21: Lynda Hardman , Jacco van Ossenbruggen , Lloyd Rutledge , Dick C. A. Bulterman, Hypermedia: the link with time, ACM Computing Surveys (CSUR), v.31 n.4es, Dec. 1999 [doi>10.1145/345966.346018]
• 22: M. Korkeaaho, "Context-Aware Applications Survey", Department of Computer Science, Helsinki University of Technology, April 2000, http://www.hut.fi/Eckorkeaa/doc/context-aware.html.
• 23: Sue Long , Rob Kooper , Gregory D. Abowd , Christopher G. Atkeson, Rapid prototyping of mobile context-aware applications: the Cyberguide case study, Proceedings of the 2nd annual international conference on Mobile computing and networking, p.97-107, November 1996, Rye, New York, United States [doi>10.1145/236387.236412]
• 24: Norman Meyrowitz, Intermedia: The architecture and construction of an object-oriented hypemedia system and applications framework, Conference proceedings on Object-oriented programming systems, languages and applications, p.186-201, September 29-October 02, 1986, Portland, Oregon, United States
• 25: E. Not, D. Petrelli, M. Sarini, O. Stock, C. Strapparava, and M. Zancanaro, "Hypernavigation in the physical space: Adapting presentations to the user and to the situational context", The New Review of Hypermedia and Multimedia, 4:33--45, 1998.
• 26: Teresa Romão , Eduardo Dias , José Danado , Nuno Correia , Adelaide Trabuco , Carlos Santos , Rossana Santos , Edmundo Nobre , António Câmara , Luís Romero, Augmenting reality with geo-referenced information for environmental management, Proceedings of the 10th ACM international symposium on Advances in geographic information systems, November 08-09, 2002, McLean, Virginia, USA [doi>10.1145/585147.585184]
• 27: Nitin Sawhney , David Balcom , Ian Smith, HyperCafe: narrative and aesthetic properties of hypervideo, Proceedings of the the seventh ACM conference on Hypertext, p.1-10, March 16-20, 1996, Bethesda, Maryland, United States [doi>10.1145/234828.234829]
• 28: Daniel Schwabe , Gustavo Rossi, The object-oriented hypermedia design model, Communications of the ACM, v.38 n.8, p.45-46, Aug. 1995 [doi>10.1145/208344.208354]
• 29: A. Steed, I. MacColl, C. Randell, B. Brown, and others, "The Use of Spacial Models in Mixed-Reality Systems (Draft)", Technical Report Equator-02-040, UCL, Glasgow, October 2002.
• 30: Hideyuki Tamura , Hiroyuki Yamamoto , Akihiro Katayama, Mixed Reality: Future Dreams Seen at the Border between Real and Virtual Worlds, IEEE Computer Graphics and Applications, v.21 n.6, p.64-70, November 2001 [doi>10.1109/38.963462]
• 31: Hiroyuki Tarumi , Ken Morishita , Yusuke Ito , Yahiko Kambayashi, Communication through virtual active objects overlaid onto the real world, Proceedings of the third international conference on Collaborative virtual environments, p.155-164, September 2000, San Francisco, California, United States [doi>10.1145/351006.351034]
• 32: Bruce Thomas , Ben Close , John Donoghue , John Squires , Phillip De Bondi , Wayne Piekarski, First Person Indoor/Outdoor Augmented Reality Application: ARQuake, Personal and Ubiquitous Computing, v.6 n.1, p.75-86, February 2002 [doi>10.1007/s007790200007]
• 33: Polle T. ZelIweger, "Scripted Documents: A Hypermedia Path Mechanism", ACM Hypertext 98, pp. 2--14.
• 34: Polle T. Zellweger , Bay-Wei Chang , Jock D. Mackinlay, Fluid links for informed and incremental link transitions, Proceedings of the ninth ACM conference on Hypertext and hypermedia : links, objects, time and space---structure in hypermedia systems: links, objects, time and space---structure in hypermedia systems, p.50-57, June 20-24, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276627.276633]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.4 Hypertext/Hypermedia
          Subjects: Architectures

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities; Video (e.g., tape, disk, DVI)
      H.5.4 Hypertext/Hypermedia
          Subjects: Theory; Navigation

General Terms:
Design, Experimentation, Human Factors, Languages, Theory

Keywords:
history, hypermedia interfaces, hypermedia model, mixed and augmented reality, mobile gaming and storytelling