Point&Connect: intention-based device pairing for mobile phone users

ABSTRACT

Point&Connect (P&C) offers an intuitive and resilient device pairing solution on standard mobile phones. Its operation follows the simple sequence of point-andconnect: when a user plans to pair her mobile phone with another device nearby, she makes a simple hand gesture that points her phone towards the intended target. The system will capture the user's gesture, understand the target selection intention, and complete the device pairing. P&C is intention-based, intuitive, and reduces user efforts in device pairing. The main technical challenge is to come up with a simple system technique to effectively capture and understand the intention of the user, and pick the right device among many others nearby. It should further work on any mobile phones or small devices without relying on infrastructure or special hardware. P&C meets this challenge with a novel collaborative scheme to measure maximum distance change based on acoustic signals. Using only a speaker and a microphone, P&C can be implemented solely in user-level software and work on COTS phones. P&C adds additional mechanisms to improve resiliency against imperfect user actions, acoustic disturbance, and even certain malicious attacks. We have implemented P&C in Windows Mobile phones and conducted extensive experimental evaluation, and showed that it is a cool and effective way to perform device pairing.

REFERENCES

• 1: Bluetooth simple pairing. http://www.bluetooth.com/NR/rdonlyres/0A0B3F36-D15F-4470-85A6-F2CCFA26F%70F/0/SimplePairing_WP_V10r00.pdf.
• 2: QR code. http://www.nttdocomo.co.jp/english/service/imode/make/content/barcode/t%ool/.
• 3: Semacode. http://semacode.com/.
• 4: WiFi Protected Setup. http://www.wi-fi.org/wifi-protected-setup.
• 5: Windows Connect Now Technology. http://www.microsoft.com/windowsxp/using/networking/getstarted/windowsconnectnow.mspx.
• 6: Wireless USB Association Models. http://www.usb.org/developers/wusb/.
• 7: Atul Adya , Paramvir Bahl , Ranveer Chandra , Lili Qiu, Architecture and techniques for diagnosing faults in IEEE 802.11 infrastructure networks, Proceedings of the 10th annual international conference on Mobile computing and networking, September 26-October 01, 2004, Philadelphia, PA, USA [doi>10.1145/1023720.1023724]
• 8: D. Balfanz, D. K. Smetters, P. Stewart, and H. C. Wong. Talking to strangers: Authentication in ad-hoc wireless networks. In Proceedings of Network and Distributed System Security Symposium 2002 (NDSS '02), San Diego, CA, February 2002.
• 9: Bryan Ford , Jacob Strauss , Chris Lesniewski-Laas , Sean Rhea , Frans Kaashoek , Robert Morris, Persistent personal names for globally connected mobile devices, Proceedings of the 7th symposium on Operating systems design and implementation, November 06-08, 2006, Seattle, Washington
• 10: Michael T. Goodrich , Michael Sirivianos , John Solis , Gene Tsudik , Ersin Uzun, Loud and Clear: Human-Verifiable Authentication Based on Audio, Proceedings of the 26th IEEE International Conference on Distributed Computing Systems, p.10, July 04-07, 2006 [doi>10.1109/ICDCS.2006.52]
• 11: Ken Hinckley, Synchronous gestures for multiple persons and computers, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.149-158, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964713]
• 12: Lars Erik Holmquist , Friedemann Mattern , Bernt Schiele , Petteri Alahuhta , Michael Beigl , Hans-Werner Gellersen, Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts, Proceedings of the 3rd international conference on Ubiquitous Computing, p.116-122, September 30-October 02, 2001, Atlanta, Georgia, USA
• 13: D. Kirovski, M. Sinclair, and D. Wilson. The martini synch: Device pairing via joint quantization. In ISIT '07: Proceedings of IEEE International Symposium on Information Theory, 2007, July 2007.
• 14: Branislav Kusy , Akos Ledeczi , Xenofon Koutsoukos, Tracking mobile nodes using RF Doppler shifts, Proceedings of the 5th international conference on Embedded networked sensor systems, November 06-09, 2007, Sydney, Australia [doi>10.1145/1322263.1322267]
• 15: C. V. Lopes and P. M. Q. Aguiar. Aerial Acoustic Communications. In Proceedings of the 2001 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, 2001.
• 16: R. Mayrhofer and H. Gellersen. Shake well before use: Authentication based on accelerometer data. In A. LaMarca, M. Langheinrich, and K. N. Truong, editors, Pervasive, volume 4480 of Lecture Notes in Computer Science, pages 144--161. Springer, 2007.
• 17: R. Mayrhofer, H. Gellersen, and M. Hazas. Security by spatial reference: Using relative positioning to authenticate devices for spontaneous interaction. In Ubicomp, pages 199--216, 2007.
• 18: Jonathan M. McCune , Adrian Perrig , Michael K. Reiter, Seeing-Is-Believing: Using Camera Phones for Human-Verifiable Authentication, Proceedings of the 2005 IEEE Symposium on Security and Privacy, p.110-124, May 08-11, 2005 [doi>10.1109/SP.2005.19]
• 19: Kai Nickel , Rainer Stiefelhagen, Visual recognition of pointing gestures for human-robot interaction, Image and Vision Computing, v.25 n.12, p.1875-1884, December, 2007 [doi>10.1016/j.imavis.2005.12.020]
• 20: Chunyi Peng , Guobin Shen , Yongguang Zhang , Yanlin Li , Kun Tan, BeepBeep: a high accuracy acoustic ranging system using COTS mobile devices, Proceedings of the 5th international conference on Embedded networked sensor systems, November 06-09, 2007, Sydney, Australia [doi>10.1145/1322263.1322265]
• 21: R. Prasad and N. Saxena. Efficient device pairing using "human-comparable"synchronized audiovisual patterns. In S. M. Bellovin, R. Gennaro, A. D. Keromytis, and M. Yung, editors, ACNS, volume 5037 of Lecture Notes in Computer Science, pages 328--345, 2008.
• 22: J. Rekimoto, Y. Ayatsuka, and M. Kohno. Synctap: An interaction technique for mobile networking. In Mobile HCI, pages 104--115, 2003.
• 23: E. Rukzio. Physical Mobile Interactions: Mobile Devices as Pervasive Mediators for Interactions with the Real World. PhD thesis, University of Munich, 2006.
• 24: E. Rukzio, K. Leichtenstern, V. Callaghan, A. Schmidt, P. Holleis, and J. Chin. An experimental comparison of physical mobile interaction techniques: Touching, pointing and scanning. In Ubicomp'06: Eighth International Conference on Ubiquitous Computing, September 2006.
• 25: Nitesh Saxena , Jan-Erik Ekberg , Kari Kostiainen , N. Asokan, Secure Device Pairing based on a Visual Channel (Short Paper), Proceedings of the 2006 IEEE Symposium on Security and Privacy, p.306-313, May 21-24, 2006 [doi>10.1109/SP.2006.35]
• 26: Nitesh Saxena , Md. Borhan Uddin , Jonathan Voris, Universal device pairing using an auxiliary device, Proceedings of the 4th symposium on Usable privacy and security, July 23-25, 2008, Pittsburgh, Pennsylvania [doi>10.1145/1408664.1408672]
• 27: Greg Schmidt , Yohan Baillot , Dennis G. Brown , Erik B. Tomlin , J. Edward II Swan, Toward Disambiguating Multiple Selections for Frustum-Based Pointing, Proceedings of the 3D User Interfaces, p.87-94, March 25-29, 2006
• 28: Claudio Soriente , Gene Tsudik , Ersin Uzun, HAPADEP: Human-Assisted Pure Audio Device Pairing, Proceedings of the 11th international conference on Information Security, September 15-18, 2008, Taipei, Taiwan [doi>10.1007/978-3-540-85886-7_27]
• 29: E. Uzun, K. Karvonen, and N. Asokan. Usability analysis of secure pairing methods. http://research.nokia.com/files/NRC-TR-2007-002.pdf, 2007.
• 30: A. Varshavsky, A. Scannell, A. LaMarca, and E. de Lara. Amigo: Proximity-based authentication of mobile devices. In Ubicomp, pages 253--270, 2007.
• 31: Koji Yatani , Koiti Tamura , Keiichi Hiroki , Masanori Sugimoto , Hiromichi Hashizume, Toss-it: intuitive information transfer techniques for mobile devices, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057046]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.3 Interconnections (subsystems)

  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.1 Network Architecture and Design
          Subjects: Wireless communication
  C.3 SPECIAL-PURPOSE AND APPLICATION-BASED SYSTEMS
      Subjects: Signal processing systems

General Terms:
Design, Human Factors, Performance

Keywords:
acoustic position sensing, device pairing, gesture-based intention, identification, mobile system, point