Utility-driven assessment of data quality

ABSTRACT

Data consumers assess quality within specific business contexts or decision tasks. The same data resource may have an acceptable level of quality for some contexts but this quality may be unacceptable for other contexts. However, existing data quality metrics are mostly derived impartially, disconnected from the specific contextual characteristics. This study argues for the need to revise data quality metrics and measurement techniques to incorporate and better reflect contextual assessment. It contributes to that end by developing new metrics for assessing data quality along commonly used dimensions - completeness, validity, accuracy, and currency. The metrics are driven by data utility, a conceptual measure of the business value that is associated with the data within a specific usage context. The suggested data quality measurement framework uses utility as a scaling factor for calculating quality measurements at different levels of data hierarchy. Examples are used to demonstrate the use of utility-driven assessment in real-world data management scenarios and the broader implications for data management are discussed

REFERENCES

• 1: Ahituv, N.A. (1980). "Systematic Approach Towards Assessing the Value of Information System," MIS Quarterly, Vol.4, No.4, pp. 61--75.
• 2: Ballou, D.P. and Pazer, H.L. (1995). "Designing Information Systems to Optimize the Accuracy-timeliness Tradeoff," Information Systems Research, Vol.6, No.1, pp. 51--72.
• 3: Donald P. Ballou , Harold L. Pazer, Modeling Completeness versus Consistency Tradeoffs in Information Decision Contexts, IEEE Transactions on Knowledge and Data Engineering, v.15 n.1, p.240-243, January 2003
• 4: Donald Ballou , Richard Wang , Harold Pazer , Giri Kumar Tayi, Modeling Information Manufacturing Systems to Determine Information Product Quality, Management Science, v.44 n.4, p.462-484, April 1998 [doi>10.1287/mnsc.44.4.462]
• 5: Berger, P.D. and Nasr, N.I. (1998). "Customer Lifetime Value: Marketing Models and Applications," Journal of Interactive Marketing, Vol.12, No.1, pp. 17--30.
• 6: InduShobha N. Chengalur-Smith , Donald P. Ballou , Harold L. Pazer, The Impact of Data Quality Information on Decision Making: An Exploratory Analysis, IEEE Transactions on Knowledge and Data Engineering, v.11 n.6, p.853-864, November 1999 [doi>10.1109/69.824597]
• 7: Michael J. Davern , Robert J. Kauffman, Discovering potential and realizing value from information technology investments, Journal of Management Information Systems, v.16 n.4, p.121-143, March 2000
• 8: DeLone, W. and Mclean, E. (1992). "Information Systems Success: The Quest for the Dependent Variable," Information Systems Research, Vol.3, No.1, pp. 60--95.
• 9: Craig W. Fisher , InduShobha Chengalur-Smith , Donald P. Ballou, The Impact of Experience and Time on the Use of Data Quality Information in Decision Making, Information Systems Research, v.14 n.2, p.170-188, June 2003 [doi>10.1287/isre.14.2.170.16017]
• 10: Hanna, R., Berger, P.D., and Abendroth, L. (2005). "Optimizing Time Limits in Retail Promotions: an Email Application," Journal of the Operational Research Society, Vol. 56, pp. 15--24.
• 11: Hilton, R.W. (1981). "The Determinants of Information Value: Synthesizing Some General Results," Management Science, Vol.27, No.1, pp. 57--64.
• 12: Hufford, D. (1996). "Data Warehouse Quality", DM Review, Vol. 6 No. 1.( http: // www. dmreview. com /article_sub.cfm?articleId=1311 (accessed February 2007).
• 13: Yang W. Lee , Diane M. Strong , Beverly K. Kahn , Richard Y. Wang, AIMQ: a methodology for information quality assessment, Information and Management, v.40 n.2, p.133-146, December 2002 [doi>10.1016/S0378-7206(02)00043-5]
• 14: Lee, Y.W. and Strong, D.M. (2003). "Knowing-Why about Data Processes and Data Quality," Journal of Management Information Systems, Vol.20, No.3, pp. 13--39.
• 15: Lee, Y.W., Pipino, L., Strong, D.M., and Wang, R.Y. (2004). "Process-Embedded Data Integrity," Journal of Database Management, Vol.15, No.1, pp. 87--103.
• 16: Little, J.D.C. (1970). "Models and Managers: The Concept of a Decision Calculus," Management Science, Vol.18, pp. 466--484.
• 17: Richard C. Morey, Estimating and improving the quality of information in a MIS, Communications of the ACM, v.25 n.5, p.337-342, May 1982 [doi>10.1145/358506.358520]
• 18: Petrison, L.A., Blattberg, R.C., and Wang, P. (1997). "Database Marketing: Past, Present, and Future", Journal of Direct Marketing, Vol. 11, No.4, pp. 109--125.
• 19: Leo L. Pipino , Yang W. Lee , Richard Y. Wang, Data quality assessment, Communications of the ACM, v.45 n.4, April 2002 [doi>10.1145/505248.506010]
• 20: Thomas C. Redman , A. Blanton Godfrey, Data Quality for the Information Age, Artech House, Inc., Norwood, MA, 1997
• 21: Schmenner, R.W. (1993). Production/Operations Management, From the Inside Out (5th Edition), New York: Macmillan Publishing Company.
• 22: Shankaranarayanan, G., Ziad, M., and Wang, R.Y. (2003). "Managing Data Quality in Dynamic Decision Making Environments: An Information Product Approach," Journal of Database Management, Vol.14, No.4, pp. 14--32.
• 23: Shankaranarayanan, G. and Watts, S. (2003). "A Relevant Believable Approach for Data Quality Assessment," Proceedings of the MIT International Conference on Information Quality (ICIQ 2003), Boston, MA, pp. 178--189.
• 24: Shankaranarayanan, G. and Even, A. (2004). "Managing Metadata in Data Warehouses: Pitfalls and Possibilities," Communications of the AIS, Vol.14, pp. 247--274.
• 25: G. Shankaranarayanan , Yu Cai, Supporting data quality management in decision-making, Decision Support Systems, v.42 n.1, p.302-317, October 2006 [doi>10.1016/j.2004.12.006]
• 26: Shankaranarayanan, G., Watts, S., and Even, A. (2006). "The Role of Process Metadata and Data Quality Perceptions in Decision Making: and Empirical Framework and Investigation," Journal of Information Technology Management, Vol.17, No.1, pp. 50--67.
• 27: Shapiro C., and Varian H.R. (1999). Information Rules, Cambridge, MA: Harvard Business School Press.
• 28: Diane M. Strong , Yang W. Lee , Richard Y. Wang, Data quality in context, Communications of the ACM, v.40 n.5, p.103-110, May 1997 [doi>10.1145/253769.253804]
• 29: Y. Richard Wang , Stuart E. Madnick, A Polygen Model for Heterogeneous Database Systems: The Source Tagging Perspective, Proceedings of the 16th International Conference on Very Large Data Bases, p.519-538, August 13-16, 1990
• 30: Richard Y. Wang , Veda C. Storey , Christopher P. Firth, A Framework for Analysis of Data Quality Research, IEEE Transactions on Knowledge and Data Engineering, v.7 n.4, p.623-640, August 1995 [doi>10.1109/69.404034]
• 31: Richard Y. Wang , M. P. Reddy , Henry B. Kon, Toward quality data: an attribute-based approach, Decision Support Systems, v.13 n.3-4, p.349-372, March 1995 [doi>10.1016/0167-9236(93)E0050-N]
• 32: Richard Y. Wang , Diane M. Strong, Beyond accuracy: what data quality means to data consumers, Journal of Management Information Systems, v.12 n.4, p.5-33, March 1996
• 33: Richard Y. Wang, A product perspective on total data quality management, Communications of the ACM, v.41 n.2, p.58-65, Feb. 1998 [doi>10.1145/269012.269022]
• 34: Wixom, B.H. and Watson, H.J. (2001). "An Empirical Investigation of the Factors Affecting Data Warehousing Success," MIS Quarterly, Vol.25, No.1, pp. 17--41.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.1 Systems and Information Theory

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.7 Database Administration

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.4 System Management

Keywords:
data management, data quality, database, decision making, information products, information value, metadata, utility