SMILE: distributed middleware for event stream processing

ABSTRACT

In this paper, we describe the SMILE (Smart MIddleware, Light Ends) system which is one of the earliest systems built in the area of distributed event stream processing. SMILE unites the "publish-subscribe" model of messaging middleware with the "continuous query" model of database systems. In SMILE, information producers, which may be sensors, applications, or databases, generate streams of events, such as RFID data, news items or stock trades; consumers specify stateful subscriptions to derived views, such as "individuals trading top 5 total volume of stock Y within x minutes before a major news story about the same company Y"; the SMILE system constructs and deploys a dataflow network of computations which process events from producers, compute derived views and deliver continuous and timely updates of subscribed views to consumers. Research challenges addressed by SMILE include: rigorously defining the semantics for correct state delivery; implementing this semantics in the presence of failure; distributing computations optimally over a network of multiple servers; scheduling and controlling message flow in this network; smoothly integrating this technology with other components, e.g., databases, services, user interfaces. SMILE is applicable to any "sense and respond" scenario that requires monitoring and processing of events as they are generated. Applications span multiple industries, e.g., monitoring financial opportunities and detecting potential fraud in financial industry, systems management and alerts in data centers, inventory management in RFID applications, and business performance management. In this paper, we describe the SMILE system, its architecture, services supported, and implementation details, and its use in multiple application scenarios.

REFERENCES

• 1: O. Damani and R. Strom, "Smart Middleware and Light Ends for Simplifying Data Integration", IEEE International Conference on Information Reuse and Integration, 2006.
• 2: R. Strom, "Technical Challenges and Future Directions for Event-Based Systems", Panel Presentation, 4 th International Workshop on Distributed Event-Based Systems, June 2005.
• 3: R. Strom, "Fault-Tolerance in the SMILE Relational Subscription System", Proceedings of the International Workshop on Distributed Event-Based Systems, May 2004.
• 4: An Autonomic Messaging Middleware with Stateful Stream Transformation, Proceedings of the First International Conference on Autonomic Computing (ICAC'04), p.316-318, May 17-18, 2004
• 5: Yuhui Jin , Rob Strom, Relational subscription middleware for Internet-scale publish-subscribe, Proceedings of the 2nd international workshop on Distributed event-based systems, June 08-08, 2003, San Diego, California [doi>10.1145/966618.966638]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications

General Terms:
Design, Experimentation, Theory

Keywords:
continuous queries, distributed systems, event processing, fault-tolerance, publish-subscribe systems, streaming systems