Isolation points: Creating performance-robust enterprise systems

ABSTRACT

This article explores a performance isolation-based approach to creating robust distributed applications. For each application, the approach is to understand the performance dependencies that pervade it and then impose constraints on the possible ‘spread’ of such dependencies through the application. The mechanisms used for this purpose, termed isolation points, are software abstractions inserted at key program locations: (1) in application interfaces, (2) in middleware implementations for making remote requests, and (3) in the system interfaces used by middleware and applications. This article demonstrates the utility of isolation points by using them to implement higher level abstractions that improve the performance-robustness of representative enterprise applications. The I-Queue abstraction uses isolation points to implement performance-robust messaging, targeting the message queues used in distributed enterprise codes. By appropriately orchestrating message dispatching, I-Queue can achieve an improvement of 16--32% in dispatched message locality based on traces obtained from the large-scale e-Pricing® search engine operated by Worldspan L.P.

REFERENCES

• 1: Aristotle research group. JABA: Java architecture for bytecode analysis. http://www.cc.gatech.edu/aristotle/Tools/jaba.html. (10/04/04).
• 2: Sandip Agarwala , Fernando Alegre , Karsten Schwan , Jegannathan Mehalingham, E2EProf: Automated End-to-End Performance Management for Enterprise Systems, Proceedings of the 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, p.749-758, June 25-28, 2007 [doi>10.1109/DSN.2007.38]
• 3: S. Agarwala , Yuan Chen , D. Milojicic , K. Schwan, QMON: QoS- and Utility-Aware Monitoring in Enterprise Systems, Proceedings of the 2006 IEEE International Conference on Autonomic Computing, p.124-133, June 12-16, 2006 [doi>10.1109/ICAC.2006.1662390]
• 4: Marcos K. Aguilera , Jeffrey C. Mogul , Janet L. Wiener , Patrick Reynolds , Athicha Muthitacharoen, Performance debugging for distributed systems of black boxes, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 5: James Aweya , Michel Ouellette , Delfin Y. Montuno , Bernard Doray , Kent Felske, An adaptive load balancing scheme for web servers, International Journal of Network Management, v.12 n.1, p.3-39, January/February 2002 [doi>10.1002/nem.421]
• 6: Thomas Ball , Ella Bounimova , Byron Cook , Vladimir Levin , Jakob Lichtenberg , Con McGarvey , Bohus Ondrusek , Sriram K. Rajamani , Abdullah Ustuner, Thorough static analysis of device drivers, Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006, April 18-21, 2006, Leuven, Belgium
• 7: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 8: Sapan Bhatia , Charles Consel , Anne-Francoise Le Meur , Calton Pu, Automatic Specialization of Protocol Stacks, Proceedings of the 29th Annual IEEE International Conference on Local Computer Networks, p.152-159, November 16-18, 2004 [doi>10.1109/LCN.2004.28]
• 9: James F. Bowring , James M. Rehg , Mary Jean Harrold, Active learning for automatic classification of software behavior, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 10: Cai, Z., Chen, Y., Kumar, V., Milojicic, D. S., and Schwan, K. 2007. Automated availability management driven by business policies. In Integr. Netw. Manage. IEEE, 264--273.
• 11: George Candea , James Cutler , Armando Fox, Improving availability with recursive microreboots: a soft-state system case study, Performance Evaluation, v.56 n.1-4, p.213-248, March 2004 [doi>10.1016/j.peva.2003.07.007]
• 12: Valeria Cardellini , Emiliano Casalicchio , Michele Colajanni , Philip S. Yu, The state of the art in locally distributed Web-server systems, ACM Computing Surveys (CSUR), v.34 n.2, p.263-311, June 2002 [doi>10.1145/508352.508355]
• 13: Miguel Castro , Peter Druschel , Anne-Marie Kermarrec , Animesh Nandi , Antony Rowstron , Atul Singh, SplitStream: high-bandwidth multicast in cooperative environments, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 14: Mike Y. Chen , Emre Kiciman , Eugene Fratkin , Armando Fox , Eric Brewer, Pinpoint: Problem Determination in Large, Dynamic Internet Services, Proceedings of the 2002 International Conference on Dependable Systems and Networks, p.595-604, June 23-26, 2002
• 15: James Clause , Wanchun Li , Alessandro Orso, Dytan: a generic dynamic taint analysis framework, Proceedings of the 2007 international symposium on Software testing and analysis, July 09-12, 2007, London, United Kingdom [doi>10.1145/1273463.1273490]
• 16: Ira Cohen , Moises Goldszmidt , Terence Kelly , Julie Symons , Jeffrey S. Chase, Correlating instrumentation data to system states: a building block for automated diagnosis and control, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.16-16, December 06-08, 2004, San Francisco, CA
• 17: Ira Cohen , Steve Zhang , Moises Goldszmidt , Julie Symons , Terence Kelly , Armando Fox, Capturing, indexing, clustering, and retrieving system history, Proceedings of the twentieth ACM symposium on Operating systems principles, October 23-26, 2005, Brighton, United Kingdom
• 18: Crispin Cowan , Shanwei Cen , Jonathan Walpole , Calton Pu, Adaptive methods for distributed video presentation, ACM Computing Surveys (CSUR), v.27 n.4, p.580-583, Dec. 1995 [doi>10.1145/234782.234794]
• 19: Grzegorz Czajkowski, Application isolation in the Java Virtual Machine, Proceedings of the 15th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, p.354-366, October 2000, Minneapolis, Minnesota, United States
• 20: Greg Eisenhauer , Fabián E. Bustamante , Karsten Schwan, Event Services for High Performance Computing, Proceedings of the 9th IEEE International Symposium on High Performance Distributed Computing, p.113, August 01-04, 2000
• 21: Armando Fox , Emre Kiciman , David Patterson, Combining statistical monitoring and predictable recovery for self-management, Proceedings of the 1st ACM SIGSOFT workshop on Self-managed systems, p.49-53, October 31-November 01, 2004, Newport Beach, California [doi>10.1145/1075405.1075415]
• 22: Ada Gavrilovska , Karsten Schwan , Van Oleson, Adaptable Mirroring in Cluster Servers, Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing, p.3, August 07-09, 2001
• 23: Ada Gavrilovska , Karsten Schwan , Van Oleson, A Practical Approach for Zero' Downtime in an Operational Information System, Proceedings of the 22 nd International Conference on Distributed Computing Systems (ICDCS'02), p.345, July 02-05, 2002
• 24: Gray, J. 1985. Why do computers stop and what can be done about it? Tech. Rep. TR.85.7, Tandem.
• 25: Gu, W., Eisenhauer, G., Schwan, K., and Vetter, J. S. 1988. Falcon: On-line monitoring and steering of parallel programs. Concurrency: Practice and Experience 10, 9, 699--736.
• 26: William G. J. Halfond , Alessandro Orso, Automated identification of parameter mismatches in web applications, Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, November 09-14, 2008, Atlanta, Georgia [doi>10.1145/1453101.1453126]
• 27: Graham Hamilton , Michael L. Powell , James G. Mitchell, Subcontract: a flexible base for distributed programming, Proceedings of the fourteenth ACM symposium on Operating systems principles, p.69-79, December 05-08, 1993, Asheville, North Carolina, United States
• 28: James E. Hanson , Ian Whalley , David M. Chess , Jeffrey O. Kephart, An Architectural Approach to Autonomic Computing, Proceedings of the First International Conference on Autonomic Computing, p.2-9, May 17-18, 2004
• 29: Qi He , Karsten Schwan, IQ-RUDP: Coordinating Application Adaptation with Network Transport, Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing, p.369, July 24-26, 2002
• 30: Jasmina Jancic , Christian Poellabauer , Karsten Schwan , Mathhew Wolf , Neil Bright, dproc - Extensible Run-Time Resource Monitoring for Cluster Applications, Proceedings of the International Conference on Computational Science-Part II, p.894-903, April 21-24, 2002
• 31: Guofei Jiang , Haifeng Chen , Cristian Ungureanu , Kenji Yoshihira, Multi-resolution Abnormal Trace Detection Using Varied-length N-grams and Automata, Proceedings of the Second International Conference on Automatic Computing, p.111-122, June 13-16, 2005 [doi>10.1109/ICAC.2005.42]
• 32: Mick Jordan , Grzegorz Czajkowski , Kirill Kouklinski , Glenn Skinner, Extending a J2EE™ server with dynamic and flexible resource management, Proceedings of the 5th ACM/IFIP/USENIX international conference on Middleware, October 18-22, 2004, Toronto, Canada
• 33: Jeffrey O. Kephart, Research challenges of autonomic computing, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062464]
• 34: Rajnish Kumar , Matthew Wolenetz , Bikash Agarwalla , JunSuk Shin , Phillip Hutto , Arnab Paul , Umakishore Ramachandran, DFuse: a framework for distributed data fusion, Proceedings of the 1st international conference on Embedded networked sensor systems, November 05-07, 2003, Los Angeles, California, USA [doi>10.1145/958491.958505]
• 35: Vibhore Kumar , Brian F. Cooper , Greg Eisenhauer , Karsten Schwan, iManage: policy-driven self-management for enterprise-scale systems, Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware, November 26-30, 2007, Newport Beach, California
• 36: Zhijun Lei , Nicolas D. Georganas, Adaptive video transcoding and streaming over wireless channels, Journal of Systems and Software, v.75 n.3, p.253-270, March 2005 [doi>10.1016/j.jss.2003.09.029]
• 37: Cheng Liao , Margaret Martonosi , Douglas W. Clark, Performance monitoring in a Myrinet-connected SHRIMP cluster, Proceedings of the SIGMETRICS symposium on Parallel and distributed tools, p.21-29, August 03-04, 1998, Welches, Oregon, United States [doi>10.1145/281035.281038]
• 38: Guy Lohman , Jon Champlin , Peter Sohn, Quickly Finding Known Software Problems via Automated Symptom Matching, Proceedings of the Second International Conference on Automatic Computing, p.101-110, June 13-16, 2005 [doi>10.1109/ICAC.2005.49]
• 39: Lowekamp, B., Miller, N., Karrer, R., Gross, T., and Steenkiste, P. 2003. Design, implementation, and evaluation of the remos network monitoring system. J. Grid Comput. 1, 1, 75--93.
• 40: J. P. Loyall , R. E. Schantz , J. A. Zinky , D. E. Bakken, Specifying and Measuring Quality of Service in Distributed Object Systems, Proceedings of the The 1st IEEE International Symposium on Object-Oriented Real-Time Distributed Computing, p.43, April 20-22, 1998
• 41: Mansour, M. and Mehalingham, J. 2006. private communication.
• 42: Mohamed Mansour , Karsten Schwan, I-RMI: performance isolation in information flow applications, Proceedings of the ACM/IFIP/USENIX 2005 International Conference on Middleware, p.375-389, November 01-01, 2005, Grenoble, France
• 43: Mansour, M. S., Schwan, K., and Abdelaziz, S. 2006a. I-Queue: Smart queues for service management. In Proceedings of the 4th International Conference on Service Oriented Computing (ICSOC'06). Lecture Notes in Computer Science. Springer.
• 44: Mansour, M. S., Schwan, K., and Abdelaziz, S. 2006b. I-Queue: Smart queues for service management. Tech. Rep. GIT-CERCS-06-11, CERCS.
• 45: Brian D. Noble , M. Satyanarayanan , Dushyanth Narayanan , James Eric Tilton , Jason Flinn , Kevin R. Walker, Agile application-aware adaptation for mobility, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.276-287, October 05-08, 1997, Saint Malo, France
• 46: Peyman Oreizy , Michael M. Gorlick , Richard N. Taylor , Dennis Heimbigner , Gregory Johnson , Nenad Medvidovic , Alex Quilici , David S. Rosenblum , Alexander L. Wolf, An Architecture-Based Approach to Self-Adaptive Software, IEEE Intelligent Systems, v.14 n.3, p.54-62, May 1999 [doi>10.1109/5254.769885]
• 47: Oster, S., Hastings, S. L., Langella, S., Ervin, D. W., Madduri, R., Kurc, T. M., Siebenlist, F., Foster, I., Shanbhag, K., Covitz, P. A., and Saltz, J. H. 2007. cagrid 1.0: A grid enterprise architecture for cancer research. Proceedings of the 2007 AMIA Annual Symposium.
• 48: Poellabauer, C., Schwan, K., West, R., Ganev, I., Bright, N., and Losik, G. 2000. Flexible user/kernel communication for real-time applications in Elinux. In Proceedings of the Workshop on Real Time Operating Systems and Applications and Second Real Time Linux Workshop (in conjunction with RTSS 2000).
• 49: Michael L. Powell , Barton P. Miller, Process migration in DEMOS/MP, Proceedings of the ninth ACM symposium on Operating systems principles, p.110-119, October 10-13, 1983, Bretton Woods, New Hampshire, United States
• 50: Rob Powers , Moises Goldszmidt , Ira Cohen, Short term performance forecasting in enterprise systems, Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining, August 21-24, 2005, Chicago, Illinois, USA [doi>10.1145/1081870.1081976]
• 51: Niels Provos , Chuck Lever, Scalable network I/O in Linux, Proceedings of the annual conference on USENIX Annual Technical Conference, p.38-38, June 18-23, 2000, San Diego, California
• 52: Pyarali, I., Schmidt, D. C., and Cytron, R. 2003. Techniques for enhancing real-time corba quality of service. Proc. IEEE 91, 7, 1070--1085.
• 53: Lawrence R. Rabiner, A tutorial on hidden Markov models and selected applications in speech recognition, Readings in speech recognition, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1990
• 54: Lyle Ramshaw , Akhil Sahai , Jim Saxe , Sharad Singhal, Cauldron: A Policy-Based Design Tool, Proceedings of the Seventh IEEE International Workshop on Policies for Distributed Systems and Networks, p.113-122, June 05-07, 2006 [doi>10.1109/POLICY.2006.13]
• 55: Christopher Roblee , George Cybenko, Implementing Large-Scale Autonomic Server Monitoring Using Process Query Systems, Proceedings of the Second International Conference on Automatic Computing, p.123-133, June 13-16, 2005 [doi>10.1109/ICAC.2005.34]
• 56: D. Rosu , K. Schwan , S. Yalamanchili, FARA ¾ A Framework for Adaptive Resource Allocation in Complex Real-Time Systems, Proceedings of the Fourth IEEE Real-Time Technology and Applications Symposium, p.79, June 03-05, 1998
• 57: D. Rosu , K. Schwan , S. Yalamanchili , R. Jha, On adaptive resource allocation for complex real-time applications, Proceedings of the 18th IEEE Real-Time Systems Symposium, p.320, December 03-05, 1997
• 58: M. S. Squiillante , E. D. Lazowska, Using Processor-Cache Affinity Information in Shared-Memory Multiprocessor Scheduling, IEEE Transactions on Parallel and Distributed Systems, v.4 n.2, p.131-143, February 1993 [doi>10.1109/71.207589]
• 59: Strom, R., Banavar, G., Chandra, T., Kaplan, M., Miller, K., Mukherjee, B., Sturman, D., and Ward, M. 1998. Gryphon: An information flow based approach to message brokering. In Proceedings of the International Symposium on Software Reliability Engineering (ISSRE'98).
• 60: Vijay Sundaram , Abhishek Chandra , Pawan Goyal , Prashant Shenoy , Jasleen Sahni , Harrick Vin, Application performance in the QLinux multimedia operating system, Proceedings of the eighth ACM international conference on Multimedia, p.127-136, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354448]
• 61: Uthayopas, P., Phaisithbenchapol, S., and Chongbarirux, K. 1998. Building a resources monitoring system for smile beowulf cluster. In Proceeding of the 3rd International Conference/Exhibition on High Performance Computing in Asia-Pacific Region (HPC ASIA'99).
• 62: Mark Weiser, Program slicing, Proceedings of the 5th international conference on Software engineering, p.439-449, March 09-12, 1981, San Diego, California, United States
• 63: Richard West , Karsten Schwan, Dynamic Window-Constrained Scheduling for Multimedia Applications, Proceedings of the 1999 IEEE International Conference on Multimedia Computing and Systems, p.87, June 07-11, 1999 [doi>10.1109/MMCS.1999.778145]
• 64: Matthew Wolf , Zhongtang Cai , Weiyun Huang , Karsten Schwan, SmartPointers: personalized scientific data portals in your hand, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-16, November 16, 2002, Baltimore, Maryland
• 65: Yuan, W. and Nahrstedt, K. 2004. Process group management in cross-layer adaptation. In Proceedings of the SPIE/ACM Multimedia Computing and Networking Conference (MMCN'04). 55--68.
• 66: Steve Zhang , Ira Cohen , Julie Symons , Armando Fox, Ensembles of Models for Automated Diagnosis of System Performance Problems, Proceedings of the 2005 International Conference on Dependable Systems and Networks, p.644-653, June 28-July 01, 2005 [doi>10.1109/DSN.2005.44]

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Reliability, availability, and serviceability

General Terms:
Management, Measurement, Performance, Reliability

Keywords:
Autonomic computing, dynamic behavior, performance isolation