Towards the automated social analysis of situated speech data

ABSTRACT

We present an automated approach for studying fine-grained details of social interaction and relationships. Specifically, we analyze the conversational characteristics of a group of 24 individuals over a six-month period, explore the relationship between conversational dynamics and network position, and identify behavioral correlates of tie strengths within a network. The ability to study conversational dynamics and social networks over long time scales, and to investigate their interplay with rigor, objectivity, and transparency will complement the traditional methods for scientific inquiry into social dynamics. They may also enable socially aware ubiquitous computing systems that are cognizant of and responsive to the user's engagement with her social environment.

REFERENCES

• 1: N. Baym, Y. B. Zhang, and M. C. Lin. Social interactions across media: Interpersonal communication on the internet, telephone and face-to-face. New Media and Society, 6:299--318, June 2004.
• 2: Tanzeem Choudhury , Gaetano Borriello , Sunny Consolvo , Dirk Haehnel , Beverly Harrison , Bruce Hemingway , Jeffrey Hightower , Predrag Pedja Klasnja , Karl Koscher , Anthony LaMarca , James A. Landay , Louis LeGrand , Jonathan Lester , Ali Rahimi , Adam Rea , Danny Wyatt, The Mobile Sensing Platform: An Embedded Activity Recognition System, IEEE Pervasive Computing, v.7 n.2, p.32-41, April 2008 [doi>10.1109/MPRV.2008.39]
• 3: R. Donovan. Trainable Speech Synthesis. PhD thesis, Cambridge University, 1996.
• 4: S. L. Feld. The focused organization of social ties. Am J Sociol, 86(5):1015--1035, March 1981.
• 5: James Fogarty , Scott E. Hudson , Christopher G. Atkeson , Daniel Avrahami , Jodi Forlizzi , Sara Kiesler , Johnny C. Lee , Jie Yang, Predicting human interruptibility with sensors, ACM Transactions on Computer-Human Interaction (TOCHI), v.12 n.1, p.119-146, March 2005 [doi>10.1145/1057237.1057243]
• 6: D. R. Gibson. Taking turns and talking ties: Networks and conversational interaction. Am J Sociol, 110(6):1561--97, May 2005.
• 7: S. M. Goodreau, J. A. Kitts, and M. Morris. Birds of a feather, or friend of a friend? Using exponential random graph models to investigate adolescent social networks. Demography, 45(4), November 2008.
• 8: J. A. Kitts, H. I. Thielmann, E. Gleave, and R. Wang. Survey of Social Relations and Interests Among Graduate Students. Center for Statistics and the Social Sciences, Seattle, WA, 2006.
• 9: G. Kossinets and D. J. Watts. Empirical analysis of an evolving social network. Science, 311:88--90, 2006.
• 10: Thomas Quatieri -, Discrete-time speech signal processing: principles and practice, Prentice Hall Press, Upper Saddle River, NJ, 2001
• 11: B. Schilit , N. Adams , R. Want, Context-Aware Computing Applications, Proceedings of the 1994 First Workshop on Mobile Computing Systems and Applications, p.85-90, December 08-09, 1994 [doi>10.1109/WMCSA.1994.16]
• 12: S. Wasserman and K. Faust. Social Network Analysis: Methods and Applications. Cambridge University Press, 1994.
• 13: D. Wyatt, T. Choudhury, and J. Bilmes. Conversation detection and speaker segmentation in privacy sensitive situated speech data. In Proceedings of Interspeech, 2007.
• 14: D. Wyatt, T. Choudhury, and J. Bilmes. Learning hidden curved exponential random graph models to infer face-to-face interaction networks from situated speech data. In Proc. of AAAI, 2008.
• 15: D. Wyatt, T. Choudhury, and H. Kautz. Capturing spontaneous conversation and social dynamics: A privacy sensitive data collection effort. In Proc. of ICASSP, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION

  J. Computer Applications
  J.5 ARTS AND HUMANITIES

Keywords:
machine learning, social network analysis, wearable computing