Configurable isolation: building high availability systems with commodity multi-core processors

ABSTRACT

High availability is an increasingly important requirement for enterprise systems, often valued more than performance. Systems designed for high availability typically use redundant hardware for error detection and continued uptime in the event of a failure. Chip multiprocessors with an abundance of identical resources like cores, cache and interconnection networks would appear to be ideal building blocks for implementing high availability solutions on chip. However, doing so poses significant challenges with respect to error containment and faulty component replacement. Increasing silicon and transient fault rates with future technology scaling exacerbate the problem. This paper proposes a novel, cost-effective, architecture for high availability systems built from future multi-core processors. We propose a new chip multiprocessor architecture that provides configurable isolation for fault containment and component retirement, based upon cost-effective modifications to commodity designs. The design is evaluated for a state-of-the-art industrial fault model and the proposed architecture is shown to provide effective fault isolation and graceful degradation even when the failure rate is high.

REFERENCES

• 1: Albonesi, D.H. Selective Cache Ways: On-Demand Cache Resource Allocation. Journal of Instruction-Level Parallelism, Vol. 2, 2000.
• 2: Todd M. Austin, DIVA: a reliable substrate for deep submicron microarchitecture design, Proceedings of the 32nd annual ACM/IEEE international symposium on Microarchitecture, p.196-207, November 16-18, 1999, Haifa, Israel
• 3: Bartlett, W. and Ball, B. Tandem's Approach to Fault Tolerance. Tandem Systems Rev., vol. 4, no. 1, Feb. 1998, pp. 84--95.
• 4: David Bernick , Bill Bruckert , Paul Del Vigna , David Garcia , Robert Jardine , Jim Klecka , Jim Smullen, NonStop® Advanced Architecture, Proceedings of the 2005 International Conference on Dependable Systems and Networks (DSN'05), p.12-21, June 28-July 01, 2005 [doi>10.1109/DSN.2005.70]
• 5: Shekhar Borkar, Designing Reliable Systems from Unreliable Components: The Challenges of Transistor Variability and Degradation, IEEE Micro, v.25 n.6, p.10-16, November 2005 [doi>10.1109/MM.2005.110]
• 6: Fred A. Bower , Paul G. Shealy , Sule Ozev , Daniel J. Sorin, Tolerating Hard Faults in Microprocessor Array Structures, Proceedings of the 2004 International Conference on Dependable Systems and Networks (DSN'04), p.51, June 28-July 01, 2004
• 7: Thomas C. Bressoud , Fred B. Schneider, Hypervisor-based fault tolerance, ACM Transactions on Computer Systems (TOCS), v.14 n.1, p.80-107, Feb. 1996 [doi>10.1145/225535.225538]
• 8: Cristian Constantinescu, Trends and Challenges in VLSI Circuit Reliability, IEEE Micro, v.23 n.4, p.14-19, July 2003 [doi>10.1109/MM.2003.1225959]
• 9: Dell, T.J. A White paper on the benefit of chipkill-correct ECC for PC Server Main Memory, IBM white paper, http://www-03.ibm.com/servers/eserver/pseries/campaigns/chipkill.pdf.
• 10: Eagle Rock Alliance Ltd. Online survey results: 2001 cost of downtime. http://contingencyplanningresearch.com/2001.Survey.pdf, Aug. 2001.
• 11: M. L. Fair , C. R. Conklin , S. B. Swaney , P. J. Meaney , W. J. Clarke , L. C. Alves , I. N. Modi , F. Freier , W. Fischer , N. E. Weber, Reliability, availability, and serviceability (RAS) of the IBM eServer z990, IBM Journal of Research and Development, v.48 n.3-4, p.519-534, May 2004
• 12: Brian T. Gold , Jangwoo Kim , Jared C. Smolens , Eric S. Chung , Vasileios Liaskovitis , Eriko Nurvitadhi , Babak Falsafi , James C. Hoe , Andreas G. Nowatzyk, Truss: A Reliable, Scalable Server Architecture, IEEE Micro, v.25 n.6, p.51-59, November 2005 [doi>10.1109/MM.2005.122]
• 13: Gold, B. T., Smolens, J. C., Falsafi, B. and Hoe, J. C. The Granularity of Soft-Error Containment in Shared Memory Multiprocessors, Proceedings of The Workshop on Silicon Errors in Logic-System Effects (SELSE), 2006.
• 14: Mohamed Gomaa , Chad Scarbrough , T. N. Vijaykumar , Irith Pomeranz, Transient-fault recovery for chip multiprocessors, Proceedings of the 30th annual international symposium on Computer architecture, June 09-11, 2003, San Diego, California
• 15: John Hennessy, The Future of Systems Research, Computer, v.32 n.8, p.27-33, August 1999 [doi>10.1109/2.781631]
• 16: Joseph, R. Exploring Core Salvage Techniques for Multi-core Architectures. Workshop on High Performance Computing Reliability Issues, 2005.
• 17: Shubhendu S. Mukherjee , Michael Kontz , Steven K. Reinhardt, Detailed design and evaluation of redundant multithreading alternatives, Proceedings of the 29th annual international symposium on Computer architecture, May 25-29, 2002, Anchorage, Alaska
• 18: Nakano, J. et al. ReViveI/O: Efficient handling of I/O in highly-available rollback-recovery servers. In HPCA, 2006.
• 19: Qureshi, M. K. et al. Microarchitecture-based introspection: A technique for transientfault tolerance in microprocessors. In Proc. of 32nd Intl. Symp. on Comp. Arch. (ISCA-32), June 2005.
• 20: Parthasarathy Ranganathan , Sarita Adve , Norman P. Jouppi, Reconfigurable caches and their application to media processing, Proceedings of the 27th annual international symposium on Computer architecture, p.214-224, June 2000, Vancouver, British Columbia, Canada
• 21: Joydeep Ray , James C. Hoe , Babak Falsafi, Dual use of superscalar datapath for transient-fault detection and recovery, Proceedings of the 34th annual ACM/IEEE international symposium on Microarchitecture, December 01-05, 2001, Austin, Texas
• 22: Steven K. Reinhardt , Shubhendu S. Mukherjee, Transient fault detection via simultaneous multithreading, Proceedings of the 27th annual international symposium on Computer architecture, p.25-36, June 2000, Vancouver, British Columbia, Canada
• 23: Eric Rotenberg, AR-SMT: A Microarchitectural Approach to Fault Tolerance in Microprocessors, Proceedings of the Twenty-Ninth Annual International Symposium on Fault-Tolerant Computing, p.84, June 15-18, 1999
• 24: Premkishore Shivakumar , Michael Kistler , Stephen W. Keckler , Doug Burger , Lorenzo Alvisi, Modeling the Effect of Technology Trends on the Soft Error Rate of Combinational Logic, Proceedings of the 2002 International Conference on Dependable Systems and Networks, p.389-398, June 23-26, 2002
• 25: Premkishore Shivakumar , Stephen W. Keckler , Charles R. Moore , Doug Burger, Exploiting Microarchitectural Redundancy For Defect Tolerance, Proceedings of the 21st International Conference on Computer Design, p.481, October 13-15, 2003
• 26: Timothy J. Slegel , Robert M. Averill III , Mark A. Check , Bruce C. Giamei , Barry W. Krumm , Christopher A. Krygowski , Wen H. Li , John S. Liptay , John D. MacDougall , Thomas J. McPherson , Jennifer A. Navarro , Eric M. Schwarz , Kevin Shum , Charles F. Webb, IBM's S/390 G5 Microprocessor Design, IEEE Micro, v.19 n.2, p.12-23, March 1999 [doi>10.1109/40.755464]
• 27: Jared C. Smolens , Jangwoo Kim , James C. Hoe , Babak Falsafi, Efficient Resource Sharing in Concurrent Error Detecting Superscalar Microarchitectures, Proceedings of the 37th annual IEEE/ACM International Symposium on Microarchitecture, p.257-268, December 04-08, 2004, Portland, Oregon [doi>10.1109/MICRO.2004.19]
• 28: Daniel J. Sorin , Milo M. K. Martin , Mark D. Hill , David A. Wood, SafetyNet: improving the availability of shared memory multiprocessors with global checkpoint/recovery, Proceedings of the 29th annual international symposium on Computer architecture, May 25-29, 2002, Anchorage, Alaska
• 29: Jayanth Srinivasan , Sarita V. Adve , Pradip Bose , Jude A. Rivers, The Impact of Technology Scaling on Lifetime Reliability, Proceedings of the 2004 International Conference on Dependable Systems and Networks (DSN'04), p.177, June 28-July 01, 2004
• 30: Jayanth Srinivasan , Sarita V. Adve , Pradip Bose , Jude A. Rivers, The Case for Lifetime Reliability-Aware Microprocessors, Proceedings of the 31st annual international symposium on Computer architecture, p.276, June 19-23, 2004, München, Germany
• 31: Jayanth Srinivasan , Sarita V. Adve , Pradip Bose , Jude A. Rivers, Exploiting Structural Duplication for Lifetime Reliability Enhancement, Proceedings of the 32nd annual international symposium on Computer Architecture, p.520-531, June 04-08, 2005
• 32: Karthik Sundaramoorthy , Zach Purser , Eric Rotenburg, Slipstream processors: improving both performance and fault tolerance, Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, p.257-268, November 2000, Cambridge, Massachusetts, United States
• 33: T. N. Vijaykumar , Irith Pomeranz , Karl Cheng, Transient-fault recovery using simultaneous multithreading, Proceedings of the 29th annual international symposium on Computer architecture, May 25-29, 2002, Anchorage, Alaska
• 34: Roland E. Wunderlich , Thomas F. Wenisch , Babak Falsafi , James C. Hoe, SMARTS: accelerating microarchitecture simulation via rigorous statistical sampling, Proceedings of the 30th annual international symposium on Computer architecture, June 09-11, 2003, San Diego, California
• 35: SPEC Benchmark Suite. http://www.spec.org and http://www.spec.org/cpu/analysis/memory/
• 36: International Technology Roadmap for Semiconductors. http://www.itrs.net/
• 37: Falcon, A. Faraboschi, P., and Ortega, D. Combining Simulation and Virtualization through Dynamic Sampling. ISPASS-2007.
• 38: Foxton Technology, http://www.intel.com/technology/magazine/computing/foxton-technology-0905.htm
• 39: Luiz André Barroso , Kourosh Gharachorloo , Robert McNamara , Andreas Nowatzyk , Shaz Qadeer , Barton Sano , Scott Smith , Robert Stets , Ben Verghese, Piranha: a scalable architecture based on single-chip multiprocessing, Proceedings of the 27th annual international symposium on Computer architecture, p.282-293, June 2000, Vancouver, British Columbia, Canada
• 40: Poonacha Kongetira , Kathirgamar Aingaran , Kunle Olukotun, Niagara: A 32-Way Multithreaded Sparc Processor, IEEE Micro, v.25 n.2, p.21-29, March 2005 [doi>10.1109/MM.2005.35]
• 41: Tendler, J. M., Dodson, J. S., Fields Jr., J. S., Le, H., and Sinharoy, B. IBM Power4 system microarchitecture. IBM Journal of Research and Development, 46(1):5--26, 2002.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Reliability, availability, and serviceability

General Terms:
Design, Performance, Reliability

Keywords:
chip multiprocessors, fault isolation, high availability