Automatic generation of data processing workflows for transportation modeling

ABSTRACT

Scientists, economists, and planners in government, industry and academia spend much of their time accessing, integrating, and analyzing data. However, many of their studies are one-of-a-kind with little sharing and reuse for subsequent endeavors. The Argos project seeks to improve the productivity of analysts by providing a framework that encourages reuse of data sources and data processing operations, and by developing tools to generate data processing workflows.

In this paper, we present an approach to automatically generate data processing workflows. First, we define a methodology for assigning formal semantics to data and operations according to a domain ontology, which allows sharing and reuse. Specifically, we define data contents using relational descriptions in an expressive logic. Second, we develop a novel planner that uses relational subsumption to connect the output of a data processing operation with the input of another. Our modeling methodology has the significant advantage that the planner can automatically insert adaptor operations wherever necessary to bridge the inputs and outputs of operations in the workflow. We have implemented the approach in a transportation modeling domain.

REFERENCES

• 1: Serge Abiteboul , Oliver M. Duschka, Complexity of answering queries using materialized views, Proceedings of the seventeenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems, p.254-263, June 01-04, 1998, Seattle, Washington, United States [doi>10.1145/275487.275516]
• 2: José Luis Ambite , Yigal Arens , Eduard Hovy , Andrew Philpot , Luis Gravano , Vasileios Hatzivassiloglou , Judith Klavans, Simplifying Data Access: The Energy Data Collection Project, Computer, v.34 n.2, p.47-54, February 2001
• 3: Jose Luis Ambite , Genevieve Giuliano , Peter Gordon , Andreas Harth , LanLan Wang , Qisheng Pan , Stefan Decker, Argos: dynamic composition of web services for goods movement analysis and planning, Proceedings of the 2004 annual national conference on Digital government research, p.1-2, May 24-26, 2004, Seattle, WA
• 4: Jose Luis Ambite , Genevieve Giuliano , Peter Gordon , Qisheng Pan , Sandipan Bhattacharjee, Integrating heterogeneous data sources for better freight flow analysis and planning, Proceedings of the 2002 annual national conference on Digital government research, p.1-12, May 19-22, 2002, Los Angeles, California
• 5: José Luis Ambite , Craig A. Knoblock, Flexible and scalable cost-based query planning in mediators: a transformational approach, Artificial Intelligence, v.118 n.1-2, p.115-161, April 2000 [doi>10.1016/S0004-3702(00)00003-5]
• 6: Yigal Arens , Craig A. Knoblock , Wei-Min Shen, Query reformulation for dynamic information integration, Journal of Intelligent Information Systems, v.6 n.2-3, p.99-130, June 1996 [doi>10.1007/BF00122124]
• 7: Patricia G. Baker , Andy Brass , Sean Bechhofer , Carole A. Goble , Norman W. Paton , Robert Stevens, TAMBIS: Transparent Access to Multiple Bioinformatics Information Sources, Proceedings of the 6th International Conference on Intelligent Systems for Molecular Biology, p.25-34, July 01, 1998
• 8: Jim Blythe , Ewa Deelman , Yolanda Gil, Automatically Composed Workflows for Grid Environments, IEEE Intelligent Systems, v.19 n.4, p.16-23, July 2004 [doi>10.1109/MIS.2004.24]
• 9: Hans Chalupsky , Thomas A. Russ, WhyNot: debugging failed queries in large knowledge bases, Eighteenth national conference on Artificial intelligence, p.870-877, July 28-August 01, 2002, Edmonton, Alberta, Canada
• 10: Ashok K. Chandra , Philip M. Merlin, Optimal implementation of conjunctive queries in relational data bases, Proceedings of the ninth annual ACM symposium on Theory of computing, p.77-90, May 04-04, 1977, Boulder, Colorado, United States [doi>10.1145/800105.803397]
• 11: Oliver Michael Duschka , Michael R. Genesereth, Query planning and optimization in information integration, Stanford University, Stanford, CA, 1998
• 12: M. R. Genesereth. Knowledge interchange format. In Proceedings of the 2nd International Conference on Principles of Knowledge Representation and Reasoning, 1991.
• 13: G. Giuliano, P. Gordon, Q. Pan, J. Park, and L. Wang. Estimating freight flows for metropolitan highway networks using secondary data sources. In Proceedings of Transportation Research Board Commodity Flow Survey Conference, Transportation Research Circular, E-C088, July 2005.
• 14: P. Gordon and Q. Pan. Assembling and processing freight shipment data: developing a gis-based origin-destination matrix for southern california freight flows. Final report of METRANS research project 99--25, University of Southern California, 2001. www.metrans.org/research/final/99--25_Final.pdf
• 15: Alon Y. Halevy, Answering queries using views: A survey, The VLDB Journal — The International Journal on Very Large Data Bases, v.10 n.4, p.270-294, December 2001 [doi>10.1007/s007780100054]
• 16: I. Horrocks, U. Sattler, S. Tessaris, and S. Tobies. How to decide query containment under constraints using a description logic. In Proceedings of the 7th International Conference on Logic for Programming and Automated Reasoning (LPAR'2000), 2000.
• 17: Jihie Kim , Marc Spraragen , Yolanda Gil, An intelligent assistant for interactive workflow composition, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964466]
• 18: C. A. Knoblock. Building a planner for information gathering: A report from the trenches. In 3rd International Conference on Artificial Intelligence Planning Systems, Edinburgh, Scotland, 1996.
• 19: R. MacGregor. A deductive pattern matcher. In Proceedings of the 7th National Conference on Artificial Intelligence, Saint Paul, MN, 1988.
• 20: Robert M. MacGregor, A description classifier for the predicate calculus, Proceedings of the twelfth national conference on Artificial intelligence (vol. 1), p.213-220, October 1994, Seattle, Washington, United States
• 21: S. McIlraith and T. C. Son. Adapting Golog for composition of semantic web services. In Proceedings of the Eighth International Conference on Knowledge Representation and Reasoning (KR2002), 2002.
• 22: J. S. Penberthy and D. S. Weld. UCPOP: A sound, complete, partial order planner for ADL. In Third International Conference on Principles of Knowledge Representation and Reasoning, Cambridge, MA, 1992.
• 23: PowerLoom. The Power Loom knowledge representation & reasoning system. www.isi.edu/isd/LOOM/PowerLoom, 2003.
• 24: Wanda Pratt , Marti A. Hearst , Lawrence M. Fagan, A knowledge-based approach to organizing retrieved documents, Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence, p.80-85, July 18-22, 1999, Orlando, Florida, United States
• 25: Y. Sheffi. Urban Transportation Networks; Equilibrium Analysis with Mathematical Programming Methods. Prentice Hall, New Jersey, 1985.
• 26: E. Sirin, B. Parsia, D. Wu, J. Hendler, and D. Nau. HTN planning for web service composition using SHOP2. Web Semantics, 1(4):377--396, 2004.
• 27: R. D. Stevens, A. J. Robinson, and C. A. Goble. myGrid: personalised bioinformatics on the information grid. Bioinformatics, 19(1):302--304, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.5 Heterogeneous Databases

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.6 Logic Programming

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services
          Subjects: Web-based services

Keywords:
information integration, knowledge representation, logic, web service composition, workflow