Visualizing real-time language-based feedback on teamwork behavior in computer-mediated groups

ABSTRACT

While most collaboration technologies are concerned with supporting particular tasks such as workflows or meetings, many work groups do not have the teamwork skills essential to effective collaboration. One way to improve teamwork is to provide dynamic feedback generated by automated analyses of behavior, such as language use. Such feedback can lead members to reflect on and subsequently improve their collaborative behavior, but might also distract from the task at hand. We have experimented with GroupMeter - a chat-based system that presents visual feedback on team members' language use. Feedback on proportion of agreement words and overall word count was presented using two different designs. When receiving feedback, teams in our study expressed more agreement in their conversations and reported greater focus on language use as compared to when not receiving feedback. This suggests that automated, real-time linguistic feedback can elicit behavioral changes, offering opportunities for future research.

REFERENCES

• 1: Bales, R.F.,&Cohen, S.P. (1979). SYMLOG: A system for the multiple level observation of groups. NY: Free Press.
• 2: Jacob T. Biehl , William T. Baker , Brian P. Bailey , Desney S. Tan , Kori M. Inkpen , Mary Czerwinski, Impromptu: a new interaction framework for supporting collaboration in multiple display environments and its field evaluation for co-located software development, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357200]
• 3: Bergstrom, T.,&Karahalios, K. (2007). Seeing More: Visualizing Audio Cues. Proc. of INTERACT'07, 29--42.
• 4: Bosworth, K. (1994). Developing collaborative skills in college students. New Directions for Teaching and Learning, 59, 25--31.
• 5: Bransford, J. (2000). How people learn: Brain, mind, experience, and school. Washington: Nat'l Academy Press.
• 6: J. J. Cadiz , Gina Venolia , Gavin Jancke , Anoop Gupta, Designing and deploying an information awareness interface, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA [doi>10.1145/587078.587122]
• 7: Cassell, J.,&Tversky, D. (2005). The language of online intercultural community formation. JCMC, 10(2).
• 8: Kwangsu Cho , Tingting Rachel Chung , William R. King , Christian Schunn, Peer-based computer-supported knowledge refinement: an empirical investigation, Communications of the ACM, v.51 n.3, p.83-88, March 2008 [doi>10.1145/1325555.1325571]
• 9: Chung, C.K.,&Pennebaker, J.W. (2007). The psychological function of function words. In K. Fiedler (Ed.), Social communication: Frontiers of social psychology (343--359). New York: Psychology Press.
• 10: Cummings J., Finholt, T., Foster, I.,&Kesselman, C. (2008). Beyond being there: A blueprint for advancing the design, development, and evaluation of virtual organizations. Building Effective Virtual Organizations workshop.
• 11: Joan Morris DiMicco , Anna Pandolfo , Walter Bender, Influencing group participation with a shared display, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031713]
• 12: Joan Morris DiMicco , Katherine J. Hollenbach , Anna Pandolfo , Walter Bender, The impact of increased awareness while face-to-face, Human-Computer Interaction, v.22 n.1, p.47-96, May 2007
• 13: Duval, S.,&Wicklund, R.A. (1972). A theory of objective self-awareness. New York Academic Press.
• 14: Thomas Erickson , Wendy A. Kellogg, Social translucence: an approach to designing systems that support social processes, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.1, p.59-83, March 2000 [doi>10.1145/344949.345004]
• 15: Thomas Erickson , David N. Smith , Wendy A. Kellogg , Mark Laff , John T. Richards , Erin Bradner, Socially translucent systems: social proxies, persistent conversation, and the design of “babble”, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.72-79, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302997]
• 16: Festinger, L. (1954). A theory of social comparison processes. Human Relations, 7(2), 117--140.
• 17: Melanie J. Martin , Peter W. Foltz, Automated team discourse annotation and performance prediction using LSA, Proceedings of HLT-NAACL 2004: Short Papers on XX, p.97-100, May 02-07, 2004, Boston, Massachusetts
• 18: William W. Gaver , Jacob Beaver , Steve Benford, Ambiguity as a resource for design, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642653]
• 19: Gorman, J.C., Foltz, P.W., Kiekel, P.A., Martin, M.J.,&Cooke, N.J. (2003). Evaluation of latent semantic analysis-based measures of team communications content. Proc. Human Factors and Ergonomics Society, 424--428.
• 20: Hackman, J.R. (1990). Groups that work (and those that don't): Creating conditions for effective teamwork. San Francisco: Jossey-Bass.
• 21: Jeroen Janssen , Gijsbert Erkens , Gellof Kanselaar, Visualization of agreement and discussion processes during computer-supported collaborative learning, Computers in Human Behavior, v.23 n.3, p.1105-1125, May, 2007 [doi>10.1016/j.chb.2006.10.005]
• 22: Jeroen Janssen , Gijsbert Erkens , Gellof Kanselaar , Jos Jaspers, Visualization of participation: Does it contribute to successful computer-supported collaborative learning?, Computers & Education, v.49 n.4, p.1037-1065, December, 2007 [doi>10.1016/j.compedu.2006.01.004]
• 23: Joshi, M.,&Roséé, C.P. (2007). Using transactivity in conversation summarization in educational dialog. Proc. SLaTE'07, 53--56.
• 24: Karrie Karahalios , Tony Bergstrom, Visualizing Audio in Group Table Conversation, Proceedings of the First IEEE International Workshop on Horizontal Interactive Human-Computer Systems, p.131-134, January 05-07, 2006 [doi>10.1109/TABLETOP.2006.37]
• 25: Shipra Kayan , Susan R. Fussell , Leslie D. Setlock, Cultural differences in the use of instant messaging in Asia and North America, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180956]
• 26: Kulyk, O., Wang, J.,&Terken, J. (2006). Real-time feedback on nonverbal behaviour to enhance social dynamics in small group meetings. Proc. MLMI'05, 150--161.
• 27: Gilly Leshed , Jeffrey T. Hancock , Dan Cosley , Poppy L. McLeod , Geri Gay, Feedback for guiding reflection on teamwork practices, Proceedings of the 2007 international ACM conference on Supporting group work, November 04-07, 2007, Sanibel Island, Florida, USA [doi>10.1145/1316624.1316655]
• 28: Levelt, W.J.M. (1989). Speaking: from intention to articulation. Cambridge, MA: MIT Press.
• 29: Marcial Losada , Pedro Sanchez , Elizabeth E. Noble, Collaborative technology and group process feedback: their impact on interactive sequences in meetings, Proceedings of the 1990 ACM conference on Computer-supported cooperative work, p.53-64, October 07-10, 1990, Los Angeles, California, United States [doi>10.1145/99332.99341]
• 30: Paul P. Maglio , Christopher S. Campbell, Tradeoffs in displaying peripheral information, Proceedings of the SIGCHI conference on Human factors in computing systems, p.241-248, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332438]
• 31: Tara Matthews, Designing and evaluating glanceable peripheral displays, Proceedings of the 6th conference on Designing Interactive systems, June 26-28, 2006, University Park, PA, USA [doi>10.1145/1142405.1142457]
• 32: John McCarthy , Peter Wright , Jayne Wallace , Andy Dearden, The experience of enchantment in human–computer interaction, Personal and Ubiquitous Computing, v.10 n.6, p.369-378, September 2006 [doi>10.1007/s00779-005-0055-2]
• 33: McGrath, J.E. (1984). Groups: Interaction and performance. Englewood Cliffs, NJ: Prentice-Hall.
• 34: McLeod, P.L., Liker, J.K.,&Lobel, S.A. (1992). Process feedback in task groups: An application of goal setting. Journal of Applied Behavioral Science, 28, 15--41.
• 35: Oakley, B., Felder, R.M., Brent, R.,&Elhajj, I. (2004). Turning student groups into effective teams. Journal of Student Centered Learning, 2(1), 9--34.
• 36: Gary M. Olson , Judith S. Olson, Distance matters, Human-Computer Interaction, v.15 n.2, p.139-178, September 2000 [doi>10.1207/S15327051HCI1523_4]
• 37: Pennebaker, J.W. (2002). What our words can say about us: Toward a broader language psychology. Psychological Science Agenda, 15, 8--9.
• 38: Pennebaker, J.W., Francis, M.E.,&Booth, R.J. (2001). Linguistic Inquiry and Word Count LIWC 2001. Erlbaum.
• 39: Pennebaker, J.W.,&King, L.A. (1999). Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology, 77, 1296--1312.
• 40: Christopher Plaue , Todd Miller , John Stasko, Is a picture worth a thousand words?: an evaluation of information awareness displays, Proceedings of Graphics Interface 2004, p.117-126, May 17-19, 2004, London, Ontario, Canada
• 41: Roberts, T.S. (2005). Computer-supported collaborative learning in higher education. Hershey, PA: Idea Group.
• 42: Rummel, N.,&Spada, H. (2005). Learning to collaborate: An instructional approach to promoting collaborative problem solving in computer-mediated settings. Journal of the Learning Sciences, 14(2), 201--241.
• 43: Gerry Stahl, Group Cognition: Computer Support for Building Collaborative Knowledge (Acting with Technology), The MIT Press, 2006
• 44: Kimberly Tee , Saul Greenberg , Carl Gutwin, Providing artifact awareness to a distributed group through screen sharing, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180891]
• 45: Edward R. Tufte, The visual display of quantitative information, Graphics Press, Cheshire, CT, 1986
• 46: Gina Turner , Michael F. Schober, Feedback on Collaborative Skills in Remote Studio Design, Proceedings of the 40th Annual Hawaii International Conference on System Sciences, p.44, January 03-06, 2007 [doi>10.1109/HICSS.2007.228]
• 47: Fernanda B. Viégas , Judith S. Donath, Chat circles, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.9-16, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302981]
• 48: Chunhua Weng , John H. Gennari, Asynchronous collaborative writing through annotations, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031705]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Computer-supported cooperative work

General Terms:
Design, Experimentation, Human Factors

Keywords:
cmc, cscw, feedback visualization, linguistic analysis, peripheral displays, teamwork