Event-based systems: opportunities and challenges at exascale

ABSTRACT

Streaming data models have been shown to be useful in many applications requiring high-performance data exchange. Application-level overlay networks are a natural way to realize these applications' data flows and their internal computations, but existing middleware is not designed to scale to the data rates and low overhead computations necessary for the high performance domain. This paper describes EVPath, a middleware infrastructure that supports the construction and management of overlay networks that can be customized both in topology and in the data manipulations being performed. Extending from a previous high-performance publish-subscribe system, EVPath not only provides for the low overhead movement and in-line processing of large data volumes, but also offers the flexibility needed to support the varied data flow and control needs of alternative higher-level streaming models. We explore some of the challenges of high performance event systems, including those experienced when operating an event infrastructure used to transport IO events at the scale of hundred+ thousand nodes. Specifically, when transporting output data from a large-scale simulation running on the ORNL Cray Jaguar petascale machine, a surprising new issue seen in experimentation at scale is the potential for strong perturbation of running applications from inappropriate speeds at which IO is performed. This requires the IO system's event transport to be explicitly scheduled to constrain resource competition, in addition to dynamically setting and changing the topologies of event delivery.

REFERENCES

• 1: H. Abbasi, M. Wolf, F. Zheng, G. Eisenhauer, S. Klasky, and K. Schwan. Scalable data staging services for petascale applications. In hpdc2009, 2009.
• 2: S. Agarwala, G. Eisenhauer, and K. Schwan. Lightweight morphing support for evolving data exchanges in distributed applications. In Proc. of the 25th International Conference on Distributed Computer Systems (ICDCS-25), June 2005.
• 3: K. Birman, A. Schiper, and P. Stephenson. Lightweight Causal and Atomic Group Multicast. ACM Transactions on Computer Systems, 9(3), Aug. 1991.
• 4: D. Box, L. Cabrera, C. Critchley, F. Curbera, D. Ferguson, A. Geller, S. Graham, D. Hull, G. Kakivaya, A. Lewis, et al. Web Services Eventing (WS-Eventing). W3C Member Submission, 2006.
• 5: L. Brenna, A. Demers, J. Gehrke, M. Hong, J. Ossher, B. Panda, M. Riedewald, M. Thatte, and W. White. Cayuga: a high-performance event processing engine. In SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, pages 1100--1102, New York, NY, USA, 2007. ACM.
• 6: Z. Cai, G. Eisenhauer, Q. He, V. Kumar, K. Schwan, and M. Wolf. Iq-services: Network-aware middleware for interactive large-data applications. Concurrency & Computation. Practice and Exprience Journal, 2005.
• 7: A. Carzaniga, D. S. Rosenblum, and A. L. Wolf. Challenges for distributed event services: Scalability vs. expressiveness. In Proc. of Engineering Distributed Objects (EDO '99), ICSE 99 Workshop, May 1999.
• 8: A. Demers, J. Gehrke, M. Hong, M. Riedewald, and W. White. A general algebra and implementation for monitoring event streams. Technical Report TR2005-1997, Cornell University, 2005.
• 9: A. Demers, J. Gehrke, M. Hong, M. Riedewald, and W. White. Towards expressive publish/subscribe systems. Lecture Notes in Computer Science, 3896:627, 2006.
• 10: G. Eisenhauer, F. Bustamante, and K. Schwan. Publish-subscribe for high-performance computing. IEEE Internet Computing - Asynchronous Middleware and Services, 10(1):8--25, January 2006.
• 11: G. Eisenhauer and L. K. Daley. Fast heterogenous binary data interchange. In Proc. of the Heterogeneous Computing Workshop (HCW2000), May 3--5 2000.
• 12: P. Eugster. Type-based publish/subscribe: Concepts and experiences. ACM Trans. Program. Lang. Syst., 29(1):6, 2007.
• 13: P. Eugster, P. Felber, R. Guerraoui, and A.-M. Kerrmarec. The many faces of publish/subscribe. Tech. Report DSC-ID:200104, École Polythechnique Fédérale de Lausanne, Lausanne, France, January 2001.
• 14: I. Foster, K. Czajkowski, D. Ferguson, J. Frey, S. Graham, T. Maguire, D. Snelling, and S. Tuecke. Modeling and managing state in distributed systems: The role of OGSI and WSRF. Proceedings of the IEEE, 93(3):604--612, 2005.
• 15: M. K. Gardner, W.-c. Feng, J. S. Archuleta, H. Lin, and X. Ma. Parallel Genomic Sequence-Searching on an Ad-Hoc Grid: Experiences, Lessons Learned, and Implications. In ACM/IEEE SC\06: The International Conference on High-Performance Computing, Networking, Storage, and Analysis, Tampa, FL, November 2006. Best Paper Nominee.
• 16: A. Grimshaw, M. Morgan, D. Merrill, H. Kishimoto, A. Savva, D. Snelling, C. Smith, and D. Berry. An open grid services architecture primer. Computer, 42(2):27--34, 2009.
• 17: Hewlet-Packard. The netperf network performance benchmark, http://www.netperf.org.
• 18: Y. Huang, A. Slominski, C. Herath, and D. Gannon. Ws-messenger: A web services based messaging system for service-oriented grid computing. In 6th IEEE International Symposium on Cluster Computing and the Grid (CCGrid06), 2006.
• 19: N. Jiang, A. Quiroz, C. Schmidt, and M. Parashar. Meteor: a middleware infrastructure for content-based decoupled interactions in pervasive grid environments. Concurr. Comput.: Pract. Exper., 20(12):1455--1484, 2008.
• 20: V. Kumar, B. F. Cooper, Z. Cai, G. Eisenhauer, and K. Schwan. Resource-aware distributed stream management using dynamic overlays. In Proceedings of the 25th IEEE International Conference on Distributed Computing Systems (ICDCS-2005), 2005.
• 21: J. Lofstead, K. Schwan, S. Klasky, N. Podhorszki, and C. Jin. Flexible io and integration for scientific codes through the adaptable io system (adios). In Challenges of Large Applications in Distributed Environments (CLADE), 2008.
• 22: L. Oliker, J. Carter, michael Wehner, A. Canning, S. Ethier, A. Mirin, G. Bala, D. parks, patrick Worley Shigemune Kitawaki, and Y. Tsuda. Leading computational methods on scalar and vector hec platforms. In Proceedings of SuperComputing 2005, 2005.
• 23: O. M. G. (OMG). Notification service specification 1.0. ftp://www.omg.org/pub/doc/formal/00-06-20.pdf, June 2000.
• 24: O. M. G. (OMG). Event service specification 1.1. ftp://www.omg.org/pub/docs/formal/01-03-01.pdf, March 2001.
• 25: B. Segall and D. Arnold. Elvin has left the building: A publish/subscribe notification service with quenching. In Proc. of the AUUG (Australian users group for Unix and Open Systems) 1997 Conference, September 1997.
• 26: D. Skeen. The enterprise-capable publish-subscribe server. http://www.vitria.com.
• 27: R. Strom, G. Banavar, T. Chandra, M. Kaplan, K. Miller, B. Mukherjee, D. Sturman, and M. Ward. Gryphon: An information flow based approach to message brokering. In International Symposium on Software Reliability Engineering '98 Fast abstract, 1998.
• 28: Tibco. TIB/rendezvous. http://www.rv.tibco.com/rvwhitepaper.html.
• 29: M. Wolf, H. Abbasi, B. Collins, D. Spain, and K. Schwan. Service Augmentation for High End Interactive Data Services. In Proceedings of Cluster 2005, 2005.
• 30: M. Wolf, Z. Cai, W. Huang, and K. Schwan. Smartpointers: Personalized scientific data portals in your hand. In Proceedings of the Proceedings of the IEEE/ACM SC2002 Conference, page 20. IEEE Computer Society, 2002.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management

Keywords:
communication, events, high-performance, middleware, overlay