A3: a coding guideline for HCI+autism research using video annotation

ABSTRACT

Due to the profile of strengths and weaknesses indicative of autism spectrum disorders (ASD), technology may play a key role in ameliorating communication difficulties with this population. This paper documents coding guidelines established through cross-disciplinary work focused on facilitating communication development in children with ASD using computerized feedback. The guidelines, referred to as A3 (pronounced A-Cubed) or Annotation for ASD Analysis, define and operationalize a set of dependent variables coded via video annotation. Inter-rater reliability data are also presented from a study currently in-progress, as well as related discussion to help guide future work in this area. The design of the A3 methodology is well-suited for the examination and evaluation of the behavior of low-functioning subjects with ASD who interact with technology.

REFERENCES

• 1: AbleNet® BIGmack® communicator -- Red. http://www.ablenetinc.com/item_detail.aspx?ItemCode=10000201 Roseville, MN, 2008
• 2: Douglas G. Altman, Practical Statistics for Medical Research, Chapman & Hall/CRC, 2006
• 3: Baskett, C. B. The effect of live interactive video on the communicative behavior in children with autism. University of North Carolina at Chapel Hill, Chapel Hill, 1996.
• 4: Birkimer, J. C. and Brown, J. H. A graphical judgmental aid which summarizes obtained and chance reliability data and helps assess the believability of experimental effects. Journal of Applied Behavior Analysis, 12, 4 (Winter 1979), 523--533.
• 5: Byrt, T. How good is that agreement? Epidemiology, 7, 5 (September 1996), 561.
• 6: Cassell, J., Kopp, S., Tepper, P., Ferriman, K. and Striegnitz, K. Trading Spaces: How Humans and Humanoids use Speech and Gesture to Give Directions. John Wiley & Sons, New York, 2007.
• 7: Center for Disease Control and Prevention, C. Autism Information Center, DD, NCBDDD, CDC. http://www.cdc.gov/ncbddd/autism/ Atlanta, 2007
• 8: Clifford, J., Marcus, G. E. and School of American Research Writing Culture. University of California Press, Berkeley, California, 1986.
• 9: Sergio Venturini, Gaussian integrals, multinomial coefficients, and homogeneous functions, Advances in Applied Mathematics, v.22 n.3, p.297-302, April 1999 [doi>10.1006/aama.1998.0627]
• 10: DeThorne, L. S. and Coletto, M. Visualizing voice: Use of computerized feedback to facilitate vocalizations in children with autism. Proseminar in Speech and Hearing Science University of Illinois at Urbana-Champaign. Champaign, IL, April 2008.
• 11: Field, T., Field, T., Sanders, C. and Nadel, J. Children with Autism Display more Social Behaviors after Repeated Imitation Sessions. Autism, 5, 3 (Sep 2001), 317--323.
• 12: Gena, A., Krantz, P., McClannahan, L. and Poulson, C. Training and Generalization of Affective Behavior Displayed by Youth with Autism. Journal of Applied Behavior Analysis, 29, 3 (Fall 1996), 291--304.
• 13: Joey Hagedorn , Joshua Hailpern , Karrie G. Karahalios, VCode and VData: illustrating a new framework for supporting the video annotation workflow, Proceedings of the working conference on Advanced visual interfaces, May 28-30, 2008, Napoli, Italy [doi>10.1145/1385569.1385622]
• 14: Joshua Hailpern, Encouraging speech and vocalization in children with autistic spectrum disorder, ACM SIGACCESS Accessibility and Computing [doi>10.1145/1328567.1328576]
• 15: Hailpern, J., Karahalios, K., Halle, J., DeThorne, L. and Coletto, M. Creating a Spoken Impact: Audio and Visual Feedback for Children with Autistic Spectrum Disorder. In Proceedings of ({In Preparation}), {In Preparation}.
• 16: Halle, J. Teaching Language in the Natural Environment: An Analysis of Spontaneity. Journal of the Association for Persons with Severe Handicaps (JASH), 12, 1 (Spring 1987), 28--37.
• 17: Hayne, H., Gross, J., Hildreth, K. and Rovee-Collier, C. Repeated reminders increase the speed of memory retrieval by 3-month-old infants. Developmental Science, 3, 3 (August 2000), 312--318.
• 18: Howlin, P. An Overview of Social Behavior in Autism. Plenum, New York, NY, 1986.
• 19: Kanner, L. Autistic Disturbances of Affective Contact. V.H. Winston, 1943.
• 20: Kazdin, A. E. Single-Case Research Designs: Methods for Clinical and Applied Setting. Oxford University Press, USA, 1982.
• 21: Steven J. Kerr , Helen R. Neale , Sue V. G. Cobb, Virtual environments for social skills training: the importance of scaffolding in practice, Proceedings of the fifth international ACM conference on Assistive technologies, July 08-10, 2002, Edinburgh, Scotland [doi>10.1145/638249.638269]
• 22: Julie A. Kientz , Rosa I. Arriaga , Marshini Chetty , Gillian R. Hayes , Jahmeilah Richardson , Shwetak N. Patel , Gregory D. Abowd, Grow and know: understanding record-keeping needs for tracking the development of young children, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240830]
• 23: Koegel, R. L., Camarata, S., Koegel, L. K., Ben-Tall, A. and Smith, A. E. Increasing Speech Intelligibility in Children with Autism. Journal of Autism and Developmental Disorders, 28, 3 (June 1998), 241--251.
• 24: Lee, L. Developmental Sentence Analysis. Northwestern University Press, Evanston, IL, 1974.
• 25: Lord, C., Risi, S., Lambrecht, L., Cook, E. J., Leventhal, B., DiLavore, P., Pickles, A. and Rutter, M. The autism diagnostic observation schedule--generic: a standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30, 3 (June 2000), 205--223.
• 26: Lovaas, I. I. The Autistic Child. John Wiley & Sons, Inc, New York, 1977.
• 27: Lovaas, O. I. Teaching Individuals with Developmental Delays: Basic Intervention Techniques. PRO-ED, Inc., Austin, TX, 2003.
• 28: Luo, Y. and Baillargeon, R. Can a Self-Propelled Box Have a Goal? Psychological Science, 16, 8 (2005 2005), 601--608.
• 29: McCalla, D. D. and Clifton, R. K. Infants' means-end search for hidden objects in the absence of visual feedback. Infant Behavior and Development, 22, 2 (January 1999), 179--195.
• 30: McGee, G. G., Krantz, P. J. and McClannahan, L. E. The facilitative effects of incidental teaching on preposition use by autistic children. Journal of Applied Behavior Analysis, 18, 1 (Sprirng 1985), 17--31.
• 31: Michaud, F. and Théberge-Turmel, C. Mobile robotic toys and autism. Springer, 2002.
• 32: National Research Council. Educating Children with Autism. Division of Behavioral and Social Sciences and Education, Washington, DC: National Academy Press, 2001.
• 33: Owens, R. E. Language Development: An Introduction (7th Edition) Allyn & Bacon, Boston, MA, 2007.
• 34: Narcís Parés , Anna Carreras , Jaume Durany , Jaume Ferrer , Pere Freixa , David Gómez , Orit Kruglanski , Roc Parés , J. Ignasi Ribas , Miquel Soler , Àlex Sanjurjo, Promotion of creative activity in children with severe autism through visuals in an interactive multisensory environment, Proceedings of the 2005 conference on Interaction design and children, p.110-116, June 08-10, 2005, Boulder, Colorado [doi>10.1145/1109540.1109555]
• 35: Prizant, B. M., Schuler, A. L., Wetherby, A. M. and Rydell, P. Enhancing language and communication: Language approaches. Wiley, New York, 1997.
• 36: Rapin, I. and Dunn, M. Language disorders in children with autism. Seminars in Pediatric Neurology, 4, 2 (June 1997), 86--92
• 37: Reichle, J., Beukelman, D. and Light, J. Implementing an augmentative communication system: Exemplary strategies for beginning communicators. Brookes Publishing Company, Baltimore, MD, 2002.
• 38: Reid, D. H., Parsons, M. B., McCarn, J. E., Green, C. W., Phillips, J. F. and Schepis, M. M. Providing a more appropriate education for severely handicapped persons: increasing and validating functional classroom tasks. Journal of Applied Behavior Analysis, 18, 4 (Winter 1985), 289--301.
• 39: Retherford, K. S. Guide to Analysis of Language Transcripts. Thinking Publications, Eau Claire, Wisconsin, 1993.
• 40: Rosenblum, K. L., Zeanah, C., McDonough, S. and Muzik., M. Video-taped coding of working model of the child interviews: a viable and useful alternative to verbatim transcripts? Infant Behavior and Development, 27, 4 (December 2004), 544--549.
• 41: Russo, N., Larson, C. and Kraus, N. Audio-vocal system regulation in children with autism spectrum disorders. Experimental Brain Research, {Epub ahead of print -- 2008}({Epub ahead of print -- 2008} {Epub ahead of print -- 2008}).
• 42: Sajwaj, T., Twardosz, S. and Burke, M. Side effects of extinction procedures in a remedial preschool. Journal of Applied Behavior Analysis, 5, 2 (Summer 1972), 163--175.
• 43: Segal, L. B., Oster, H., Cohen, M., Caspi, B., Myers, M. and Brown, D. Smiling and Fussing in Seven-Month-Old Preterm and Full-Term Black Infants in the Still-Face Situation. Child Development, 66, 6 1995), 1829--1843.
• 44: Sheinkopf, S. J., Mundy, P., Oller, D. K. and Steffens, M. Vocal Atypicalities of Preverbal Autistic Children. Journal of Autism and Developmental Disorders, 30, 4 (August 2000), 345--354.
• 45: Lucy A. Suchman, Plans and situated actions: the problem of human-machine communication, Cambridge University Press, New York, NY, 1987
• 46: Tartaro, A. and Cassell, J. Playing with Virtual Peers: Bootstrapping Contingent Discourse in Children with Autism. In Proceedings of the Proceedings of International Conference of the Learning Sciences (Utrecht, Netherlands, June 24--28, 2008). ACM Press, 2008.
• 47: Wetherby, A. M., Prizant, B. M. and Hutchinson, T. A. Communicative, Social/Affective, and Symbolic Profiles of Young Children With Autism and Pervasive Developmental Disorders. American Journal of Speech--Language Pathology, 7, May (May 1998), 79--91.
• 48: Whyte, W. The social life of small urban spaces. Conservation Foundation, Washington, DC, 1980.
• 49: Woods, J. J. and Wetherby, A. M. Early Identification of and Intervention for Infants and Toddlers Who Are at Risk for Autism Spectrum Disorder. Language, Speech, and Hearing Services in Schools, 34, July 2003 (July 2003), 180--193.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

Additional Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

General Terms:
Experimentation, Human Factors, Measurement, Reliability

Keywords:
ASD, annotation, audio feedback, autism, coding, guidline, kappa, point-by-point agreement, reliability, video, visualization