The SIT book: audio as affective imagery for interactive storybooks

ABSTRACT

We describe a working prototype built as part of our continuing research focus on new document genres and the crossmodal affordances of interactive audio. Our experimental SIT (Sound-Image-Text) Book is a personal interactive reading experience that combines the look and feel of a real book -- a beautiful cover, paper pages and printed images and text -- with the rich, evocative quality of a movie soundtrack. The soundtrack is multi-track and includes music and sound effects. The SIT Book uses electric field sensors located in the book binding to sense the reader's casual book handling and fingering of the page; these sensors control the ambient audio. The particular point of the SIT Book is to explore the use of background sound to provide a sense of place, and to add affectto the experience of reading a book without interrupting the flow of the story.

REFERENCES

• 1: Silberman, Steve. "Ex Libris." Wired, July 1998.
• 2: Elizabeth D. Mynatt , Maribeth Back , Roy Want , Michael Baer , Jason B. Ellis, Designing audio aura, Proceedings of the SIGCHI conference on Human factors in computing systems, p.566-573, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274720]
• 3: Paradiso, J. and N. Gershenfeld, "Musical Applications of Electric Field Sensing." October 1995 Computer Music Journal). Also jrs.www.media.mit.edu/people/jrs/lazyfish/

INDEX TERMS

Keywords:
affect, audio books, electronic books, interactive audio, interactive books, peripheral audio, reading, sound design