Minimal-impact audio-based personal archives

ABSTRACT

Collecting and storing continuous personal archives has become cheap and easy, but we are still farfromcreating a useful, ubiquitous memory aid. We view the inconvenience to the user of being 'instrumented'as one of the key barriers to the broader development and adoption of these technologies. Audio-only recordings,however, can have minimal impact, requiring only that a device the size and weight of a cellphone be carried somewhere on the person. We have conducted some small-scale experiments on collecting continuous personal recordings of this kind, and investigating how they can be automatically analyzed and indexed, visualized, and correlated with other minimal-impact, opportunistic data feeds (such as online calendars and digital photo collections). We describe our unsupervised segmentation and clustering experiments in which we can achieve good agreement with hand-marked environment/situation labels. We al so di scuss some of the broader issues raised by this kind of work including privacy concerns,and describe our future plans to address these and other questions.

REFERENCES

• 1: AquaMinds Software.NoteTaker: An outlining program, 2003.http://www.aquaminds.com/
• 2: S. Chen and P. Gopalakrishnan. Speaker, environment and channel change detection and clustering via the Bayesian Information Criterion. In Proc. DARPA Broadcast News Transcription and Understanding Workshop 1998. http://www.nist.gov/speech/publications/darpa98/pdf/bn20.pdf
• 3: B. Clarkson, N. Sawhney, and A. Pentland Auditory context awareness via wearable computing. In Proc. Perceptual User Interfaces Workshop 1998. http://web.media.mit.edu/~nitin/NomadicRadio/PUI98/pui98.pdf.
• 4: B.P. Clarkson. Life patterns: structure from wearable sensors PhD thesis, MIT Media Lab, 2002. http://web.media.mit.edu~clarkson/thesis.pdf
• 5: Digital Innovations. The Neuros digital audio computer, 2003. http://www.neurosaudio.com/
• 6: D.P.W. Ellis and K. Lee. Features for segmenting and classifying long-duration recordings of "personal" audio. In Proc. ISCA Tutorial and Research Workshop on Statistical and Perceptual Audio Processing SAPA-04 Jeju, Korea, October 2004. http://www.ee.columbia.edu~dpwe/pubs/sapa04-persaud.pdf
• 7: M. Flynn, 2004.Personal communication.
• 8: Eric Freeman , David Gelernter, Lifestreams: a storage model for personal data, ACM SIGMOD Record, v.25 n.1, p.80-86, March 1996 [doi>10.1145/381854.381893]
• 9: Hyun Won Co.(mobiBLU).MusicMasterFM DAH-420 MP3 player,2004. http://itave.com/mp3_players/musicmaster.html
• 10: M. Lamming and M. Flynn.F orget-me-not: Intimate computing in support of human memory. In Proc. FRIEND21, 94 International Symposium on Next Generation Human Interface Meguro Gajoen, Japan, 1994. http://www.lamming.com/mik/Papers/fmn.pdf
• 11: N. Morgan, D. Baron, J. Edwards, D. Ellis, D.Gelbart, A. cJanin, T. Pfau, E.Shriberg, and A. Stolcke. Themeeting project at ICSI. In Proc. HLT pages 246--252, 2001.
• 12: A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In Advances in NIPS MIT Press, Cambridge, MA, 2001. http://citeseer.ist.psu.edu/ng01spectral.html
• 13: Nuria Oliver , Eric Horvitz, Selective perception policies for guiding sensing and computation in multimodal systems: a comparative analysis, Proceedings of the 5th international conference on Multimodal interfaces, November 05-07, 2003, Vancouver, British Columbia, Canada [doi>10.1145/958432.958442]
• 14: S. Renals and D.P. Ellis. Audio information access from meeting rooms. In Proc. ICASSP HongKong, 2003. http://www.dcs.shef.ac.uk/~sjr/pubs/2003/icassp03-mtg.html
• 15: D. Reynolds. An overview of automatic speaker recognition technology. In Proc. IEEE ICASSP-02 Orlando, FL, 2002.
• 16: K. Sjölander and J. Beskow. Wavesurfer-anopen source speech tool. In Proc. ICSLP-2000 Beijing, 2000. http://www.speech.kth.se/wavesurfer/wsurf_icslp00.pdf
• 17: Soundman. OKM IMic set with A3 adapter, 2004. http://www.outwardsound.com/product/microphones/72/
• 18: Lisa Stifelman , Barry Arons , Chris Schmandt, The audio notebook: paper and pen interaction with structured speech, Proceedings of the SIGCHI conference on Human factors in computing systems, p.182-189, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365096]
• 19: A. Stolcke, C. Wooters, N. Mirghafori, T. Pirinen, I. Bulyko, D. Gelbart, M. Graciarena, S. Otterson, B. Peskin, and M. Ostendorf. Progress in meeting recognition: The ICSI-SRI-UW spring 2004 evaluation system. In NIST 2004 Meeting Recognition Workshop Montreal, May 2004. http://www.speech.sri.com/papers/nist2004-meeting-system.ps.gz
• 20: G. Williams and D. Ellis. Speech/music discrimination based on posterior probability features. In Proc. Eurospeech-99 1999. ftp://ftp.icsi.berkeley.edu/pub/speech/papers/euro99-mussp.pdf

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

General Terms:
Algorithms

Keywords:
archives, audio, clustering, diary, recording, segmentation, sound