Chants and Orcas: semi-automatic tools for audio annotation and analysis in niche domains

ABSTRACT

The recent explosion of web-based collaborative applications in business and social media sites demonstrated the power of collaborative internet scale software. This includes the ability to access huge datasets, the ability to quickly update software, and the ability to let people around the world collaborate seamlessly. Multimedia learning techniques have the potential to make unstructured multimedia data accessible, reusable, searchable, and manageable. We present two different web-based collaborative projects: Cantillion, and the Orchive. Cantillion enables ethnomusicology scholars to listen and view data relating to chants from a variety of traditions, letting them view and interact with various pitch contour representations of the chant. The Orchive is a project to digitize over 20,000 hours of Orcinus orca (killer whale) vocalizations, recorded over a period of approximately 35 years, and provide tools to assist their study. The developed tools utilize ideas and techniques that are similar to the ones used in general multimedia domains such as sports video or news. However, their niche nature has presented us with special challenges as well as opportunities. Unlike more traditional domains where there are clearly defined objectives one of the biggest challenges has been the desire to support researchers to formulate questions and problems related to the data even when there is no clearly defined objective.

REFERENCES

• 1: Arturo Camacho , John G. Harris, Swipe: a sawtooth waveform inspired pitch estimator for speech and music, University of Florida, Gainesville, FL, 2007
• 2: O. Chappelle, B. Scholkopf, and A. Zien. Semi-Supervised Learning. MIT Press, Cambridge, MA, 2006.
• 3: V. Deecke, J. Ford, and P. Spong. Dialect change in resident killer whales (orcinus orca): implications for vocal learning and cultural transmission. Animal Behaviour, 60(5):619--638, 2000.
• 4: Roy Thomas Fielding , Richard N. Taylor, Architectural styles and the design of network-based software architectures, University of California, Irvine, 2000
• 5: J. Foote. Content-based Retrieval of Music and Audio, pages 138--147. 1997.
• 6: J. Ford. A catalogue of underwater calls produced by killer whales (orcinus orca) in british columbia. Technical Report 633, Canadian Data Report of Fisheries and Aquatic Science, 1987.
• 7: J. Ford. Acoustic behaviour of resident killer whales (orcinus orca) off vancouver island, british columbia. Canadian Journal of Zoology, 64:727--745, 1989.
• 8: J. Ford, E. G. M., and B. K. C. Killer Whales: The natural history and genealogy of Orcinus orca in British Columbia and Washington, 2nd ed. UBC, Vancouver, 2000.
• 9: A. Hauptman and et al. Informedia at trec 2003: Analyzing and searching broadcast news video. In Proc. of (VIDEO) TREC 2003, Gaithersburg, MD, 2003.
• 10: A. Hauptman and M. Witbrock. Informedia: News-on-demand Multimedia Information Acquisition and Retrieval. MIT Press, Cambridge, Mass, 1997.
• 11: T. Karp. Aspects of Orality and Formularity in Gregorian Chant. Northwestern University Press, Evanston, 1998.
• 12: Z. Kodaly. Folk Music of Hungary. Corvina Press, Budapest, 1960.
• 13: C. L. Krumhansl. Cognitive Foundations of Musical Pitch. Oxford University Press, Oxford, 1990.
• 14: K. Levy. Gregorian Chant and the Carolingians. Princeton University Press, Princeton, 1998.
• 15: L. McColl-Sylveste, Professional haXe and Neko, John Wiley & Sons, Inc., New York, NY, 2007
• 16: K. Nelson. The Art of Reciting the Koran. University of Texas Press, Austin, 1985.
• 17: Dave Thomas , David Hansson , Leon Breedt , Mike Clark , James Duncan Davidson , Justin Gehtland , Andreas Schwarz, Agile Web Development with Rails, Pragmatic Bookshelf, 2006
• 18: L. Treitler. The early history of music writing in the west. Journal of the American Musicological Society, 35, 1982.
• 19: G. Tzanetakis. Song specific bootstrapping of singing voice structure. In Proc. Int. Conf. on Multimedia and Exposition ICME, TaiPei, Taiwan, 2004. IEEE.
• 20: George Tzanetakis , Perry Cook, MARSYAS: a framework for audio analysis, Organised Sound, v.4 n.3, p.169-175, December 1999 [doi>10.1017/S1355771800003071]
• 21: G. Tzanetakis, M. Lagrange, P. Spong, and H. Symonds. Orchive: Digitizing and analyzing orca vocalizations. In Proc. of the RIAO, Large-Scale Semantic Access to Content Conference. RIAO, 2007.
• 22: B. M. Weiss, F. Ladich, P. Spong, and H. Symonds. Vocal behavior of resident killer whale matrilines with newborn calves: The role of family signatures. The Journal of the Acoustical Society of America, 119(1):627--635, 2006.
• 23: G. Wigoder and et al. Masora, The Encyclopedia of Judaism. MacMillan Publishing Company, New York, 1989.
• 24: H. Zimmermann. Untersuchungen zur Musikauffassung des rabbinischen Judentums. Peter Lang, Bern, 2000.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.m Miscellaneous

General Terms:
Algorithms, Human Factors

Keywords:
audio feature extraction, machine learning, multimedia analysis, multimedia annotation, semi-automatic annotation