MoBiS-Q: a tool for evaluating the success of mobile business services

ABSTRACT

Companies deploy mobile business services to enable efficient work processes and gain increases in productivity. However, the success of the services in fulfilling these goals depends on several factors from the usability of the service to its success in supporting the business processes of the companies. This paper reviews existing measures for the usability of services and measures for evaluating the effects of mobile business services on the productivity of the company. We discuss the usefulness of the existing measures in the mobile business context, where both mobility and work-context pose specific demands for the services. The review showed that existing measures rarely consider the great contextual variation caused by mobility of the services and the demands this poses on usability; which, in turn, affects productivity. To build a measurement tool that better meets the requirements of mobile business services, we completed case studies on two mobile business services, one used in passenger transport and the other in construction sites. Based on the understanding gained from the case studies, we propose a list of themes addressing both usability and productivity measures that work as the basis for a multidisciplinary measurement tool, MoBiS-Q.

REFERENCES

• 1: Alahuhta, P., Ahola, J. and Hakala, H. Mobilizing Business Applications -- A survey about the opportunities and challenges of mobile business applications and services in Finland. Technology Review 167/2005. Tekes, Helsinki, 2005.
• 2: Brooke, J. SUS -- A quick and dirty usability scale. Available online at http://www.usability.serco.com/trump/documents/Suschapt.doc.
• 3: Erik Brynjolfsson , Lorin M. Hitt, Beyond the productivity paradox, Communications of the ACM, v.41 n.8, p.49-55, Aug. 1998 [doi>10.1145/280324.280332]
• 4: Erik Brynjolfsson , Lorin Hitt, Paradox lost? Firm-level evidence on the returns to information systems spending, Management Science, v.42 n.4, p.541-558, April 1996 [doi>10.1287/mnsc.42.4.541]
• 5: Erik Brynjolfsson, The productivity paradox of information technology, Communications of the ACM, v.36 n.12, p.66-77, Dec. 1993 [doi>10.1145/163298.163309]
• 6: Minder Chen , Dongsong Zhang , Lina Zhou, Providing web services to mobile users: the architecture design of an m-service portal, International Journal of Mobile Communications, v.3 n.1, p.1-18, May 2005 [doi>10.1504/IJMC.2005.005870]
• 7: John P. Chin , Virginia A. Diehl , Kent L. Norman, Development of an instrument measuring user satisfaction of the human-computer interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.213-218, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57203]
• 8: Davis, F. D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, September (1989), 318--340.
• 9: Alan Dix , Tom Rodden , Nigel Davies , Jonathan Trevor , Adrian Friday , Kevin Palfreyman, Exploiting space and location as a design framework for interactive mobile systems, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.285-321, Sept. 2000 [doi>10.1145/355324.355325]
• 10: Duyshart, B, Walker, D, Mohamed, S. & Hampton, K. An example of developing a business model for ICT adoption on construction projects -- the National Museum of Australia project. Engineering, Construction and Architectural Management, 10, 3 (2003), 179--192.
• 11: Daniel Fallman, From Freedom to Involvement: On the Rhetoric of Mobility in HCI Research, Proceedings of the Proceedings of the 38th Annual Hawaii International Conference on System Sciences (HICSS'05) - Track 1, p.32.1, January 03-06, 2005 [doi>10.1109/HICSS.2005.279]
• 12: Erik Frøkjær , Morten Hertzum , Kasper Hornbæk, Measuring usability: are effectiveness, efficiency, and satisfaction really correlated?, Proceedings of the SIGCHI conference on Human factors in computing systems, p.345-352, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332455]
• 13: Judith Gebauer , Michael J. Shaw, Success Factors and Impacts of Mobile Business Applications: Results from a Mobile e-Procurement Study, International Journal of Electronic Commerce, v.8 n.3, p.19-41, Number 3/Spring 2004
• 14: Kasper Hornbík, Current practice in measuring usability: Challenges to usability studies and research, International Journal of Human-Computer Studies, v.64 n.2, p.79-102, February 2006 [doi>10.1016/j.ijhcs.2005.06.002]
• 15: ISO 9241-11. Ergonomic requirements for office work with visual display terminals -- part 11: guidance on usability (ISO/IEC 9421-11: 1998).
• 16: Kaasinen, E. User Acceptance of Mobile Services - Value, Ease of Use, Trust and Ease of Adoption. VTT Information Technology, Helsinki, 2005.
• 17: Masao Kakihara , Carsten Sørensen, Expanding the 'mobility' concept, ACM SIGGROUP Bulletin, v.22 n.3, p.33-37, December 2001 [doi>10.1145/567352.567358]
• 18: Kaplan, R. S., Norton, D. P. The Balanced Scorecard. Translating Strategy into Action. Harvard Business School Press, Boston, Massachusetts, 1996.
• 19: Kemppilä, S. and Lönnqvist, A. Subjective Productivity Measurement. The Journal of American Academy of Business, Cambridge, 2, 2 (2003), 531--537.
• 20: James R. Lewis, IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use, International Journal of Human-Computer Interaction, v.7 n.1, p.57-78, Jan.-March 1995 [doi>10.1080/10447319509526110]
• 21: Lynch, W. & Riedel, J. Measuring Employee Productivity: A guide to self-assessment tools. William M. Mercer & Institute for Health and Productivity Management, 2001.
• 22: NASA Task Load Index. Available online at http://iac.dtic.mil/hsiac/docs/TLX-UserManual.pdf.
• 23: Jakob Nielsen , Jonathan Levy, Measuring usability: preference vs. performance, Communications of the ACM, v.37 n.4, p.66-75, April 1994 [doi>10.1145/175276.175282]
• 24: Jakob Nielsen, Usability Engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 25: Rangone, A., & Renga, F. B2e mobile internet: an exploratory study of Italian applications. Business Process Management Journal, 12, 3, (2006), 330--343.
• 26: Ryu, Y. S. Development of Usability Questionnaires for Electronic Mobile Products and Decision Making Methods. Doctoral dissertation, State University, Virginia, 2005.
• 27: Ryu, Y. S., & Smith-Jackon, T. L. Reliability and Validity of the Mobile Phone Usability Questionnaire (MPUQ). Journal of Usability Studies, 2, 1, (2006), 39--53.
• 28: Sink, D. S. Productivity management: planning, measurement and evaluation, control and improvement. John Wiley & Sons, New York, 1985.
• 29: Software Usability Measurement Inventory. Information on line at http://www.ucc.ie/hfrg/questionnaires/sumi/.
• 30: Dan Steinbock, The Mobile Revolution: The Making of Mobile Services Worldwide, Kogan Page, Limited, 2007
• 31: Sørensen, C-F., Wang, A. I., & Conradi, R., Support of smart work processes in context of rich environments. In Krogstie, J., Kautz, K., & Allen, D. (eds.), IFIP TC8 Working Conference on Mobile Information Systems (MOBIS'2005), (Leeds, UK, December 6--7 2005). Springer, New York, USA, (2005), 15--30.
• 32: Uusi-Rauva, E. Yleiskatsaus mittausmenetelmiin. In Uusi- Rauva, E. (ed.) Tuottavuus -- mittaa ja menesty. Vantaa, (1996), 41--74 (in Finnish).
• 33: Wiberg, M. "Anytime, anywhere" in the context of mobile work. In Encyclopedia of Information Science and technology I-V. Idea Group, Inc. 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS

General Terms:
Economics, Human Factors, Management, Measurement, Performance

Keywords:
measurement, mobile business services, productivity, questionnaire, usability