PADS: a domain-specific language for processing ad hoc data

ABSTRACT

PADS is a declarative data description language that allows data analysts to describe both the physical layout of ad hoc data sources and semantic properties of that data. From such descriptions, the PADS compiler generates libraries and tools for manipulating the data, including parsing routines, statistical profiling tools, translation programs to produce well-behaved formats such as Xml or those required for loading relational databases, and tools for running XQueries over raw PADS data sources. The descriptions are concise enough to serve as "living" documentation while flexible enough to describe most of the ASCII, binary, and Cobol formats that we have seen in practice. The generated parsing library provides for robust, application-specific error handling.

REFERENCES

• 1: Abstract syntax description language. http://sourceforge.net/projects/asdl.
• 2: Cisco netflow. http://www.cisco.com/warp/public/732/Tech/nmp/netflow/index.shtml.
• 3: DFDL project. http://forge.gridforum.org/projects/dfdl-wg.
• 4: Erlang bit syntax. http://www.erlang.se/euc/99/binaries.ps.
• 5: Galax user manual. http://www.galaxquery.org/doc.html#manual.
• 6: Hypertext transfer protocol -- HTTP/1.1. http://www.w3.org/Protocols/rfc2616/rfc2616.html.
• 7: PADS user manual. http://www.padsproj.org/doc.html#manual.
• 8: Unicode home page. http://www.unicode.org/.
• 9: Godmar Back, DataScript - A Specification and Scripting Language for Binary Data, The ACM SIGPLAN/SIGSOFT Conference on Generative Programming and Component Engineering, p.66-77, October 06-08, 2002
• 10: J. Bell , F. Bellegarde , J. Hook , R. B. Kieburtz , A. Kotov , J. Lewis , L. McKinney , D. P. Oliva , T. Sheard , L. Tong , L. Walton , T. Zhou, Software design for reliability and reuse: a proof-of-concept demonstration, Proceedings of the conference on TRI-Ada '94, p.396-404, November 06-11, 1994, Baltimore, Maryland, United States [doi>10.1145/197694.197740]
• 11: S. Boag, D. Chamberlin, M. F. Fernández, D. Florescu, J. Robie, and J. Siméon. XQuery 1.0 An XML Query Language, W3C Working Draft, Aug 2004. http://www.w3.org/TR/xquery.
• 12: S. Chandra, N. Heintze, D. MacQueen, D. Oliva, and M. Siff. C-frontend library for SML/NJ. See cm.bell-labs.com/cm/cs/what/smlnj., 1999.
• 13: Corinna Cortes , Kathleen Fisher , Daryl Pregibon , Anne Rogers , Frederick Smith, Hancock: A language for analyzing transactional data streams, ACM Transactions on Programming Languages and Systems (TOPLAS), v.26 n.2, p.301-338, March 2004 [doi>10.1145/973097.973100]
• 14: C. Cortes and D. Pregibon. Giga mining. In KDD, 1998.
• 15: Corinna Cortes , Daryl Pregibon, Information mining platforms: an infrastructure for KDD rapid deployment, Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, p.327-331, August 15-18, 1999, San Diego, California, United States [doi>10.1145/312129.312270]
• 16: Chuck Cranor , Yuan Gao , Theodore Johnson , Vlaidslav Shkapenyuk , Oliver Spatscheck, Gigascope: high performance network monitoring with an SQL interface, Proceedings of the 2002 ACM SIGMOD international conference on Management of data, June 03-06, 2002, Madison, Wisconsin [doi>10.1145/564691.564777]
• 17: Olivier Dubuisson , Philippe Fouquart, ASN.1: communication between heterogeneous systems, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2001
• 18: M. F. Fernández, J. Siméon, B. Choi, A. Marian, and G. Sur. Implementing XQuery 1.0: The Galax experience. In VLDB, pages 1077--1080. ACM, 2003.
• 19: G. Fowler, D. Korn, S. North, and P. Vo. The AT&T AST opensource software collection. In Proceedings of the FREENIX Track 2000 Usenix Annual Technical Conference, pages 187--195, 2000.
• 20: Anna C. Gilbert , Sudipto Guha , Piotr Indyk , Yannis Kotidis , S. Muthukrishnan , Martin J. Strauss, Fast, small-space algorithms for approximate histogram maintenance, Proceedings of the thiry-fourth annual ACM symposium on Theory of computing, May 19-21, 2002, Montreal, Quebec, Canada [doi>10.1145/509907.509966]
• 21: A. C. Gilbert, Y. Kotidis, S. Muthukrishnan, and M. Strauss. How to summarize the universe: Dynamic maintenance of quantiles. In VLDB, pages 454--465, 2002.
• 22: Rick Greer, Daytona and the foruth-generation language cymbal, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.525-526, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 23: Sudipto Guha , Piotr Indyk , S. Muthukrishnan , Martin Strauss, Histogramming Data Streams with Fast Per-Item Processing, Proceedings of the 29th International Colloquium on Automata, Languages and Programming, p.681-692, July 08-13, 2002
• 24: Richard B. Kieburtz , Laura McKinney , Jeffrey M. Bell , James Hook , Alex Kotov , Jeffrey Lewis , Dino P. Oliva , Tim Sheard , Ira Smith , Lisa Walton, A software engineering experiment in software component generation, Proceedings of the 18th international conference on Software engineering, p.542-552, March 25-29, 1996, Berlin, Germany
• 25: D. G. Korn and K.-P. Vo. SFIO: Safe/fast string/file IO. In Proc. of the Summer '91 Usenix Conference, pages 235--256. USENIX, 1991.
• 26: B. Krishnamurthy and J. Rexford. Web Protocols and Practice. Addison Wesley, 2001.
• 27: Balachander Krishnamurthy , Jia Wang, On network-aware clustering of Web clients, Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, p.97-110, August 28-September 01, 2000, Stockholm, Sweden
• 28: Balachander Krishnamurthy , Craig E. Wills, Improving web performance by client characterization driven server adaptation, Proceedings of the 11th international conference on World Wide Web, May 07-11, 2002, Honolulu, Hawaii, USA [doi>10.1145/511446.511486]
• 29: Peter J. McCann , Satish Chandra, Packet types: abstract specification of network protocol messages, Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, p.321-333, August 28-September 01, 2000, Stockholm, Sweden

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.3 Language Constructs and Features
          Subjects: Input/output

General Terms:
Languages

Keywords:
data description language, domain-specific languages