Core monitors: monitoring performance in multicore processors

ABSTRACT

As we reach the limits of single-core computing, we are promised more and more cores in our systems. Modern architectures include many performance counters per core, but few or no inter-core counters. In fact, performance counters were not designed to be exploited by users, as they now are, but simply as aids for hardware debugging and testing during system creation. As such, they tend to be an "after thought" in the design, with no standardization across or within platforms. Nonetheless, given access to these counters, researchers are using them to great advantage [17]. Furthermore, evaluating counters for multicore systems has become a complex and resource consuming task. We propose a Performance Monitoring System consisting of a specialized CPU core designed to allow efficient collection and evaluation of performance data for both static and dynamic optimizations. Our system provides a transparent mechanism to change architectural features dynamically, inform the Operating System of process behaviors, and assist in profiling and debugging. For instance, a piece of hardware watching snoop packets can determine when a write-update cache coherence protocol would be helpful or detrimental to the currently running program. Our system is designed to allow the hardware to feed performance statistics back to software, allowing dynamic architectural adjustments at runtime.

REFERENCES

• 1: Brinkley Sprunt, Pentium 4 Performance-Monitoring Features, IEEE Micro, v.22 n.4, p.72-82, July 2002 [doi>10.1109/MM.2002.1028478]
• 2: Walter Binder, Portable and accurate sampling profiling for Java, Software—Practice & Experience, v.36 n.6, p.615-650, May 2006 [doi>10.1002/spe.v36:6]
• 3: Nathan L. Binkert , Ronald G. Dreslinski , Lisa R. Hsu , Kevin T. Lim , Ali G. Saidi , Steven K. Reinhardt, The M5 Simulator: Modeling Networked Systems, IEEE Micro, v.26 n.4, p.52-60, July 2006 [doi>10.1109/MM.2006.82]
• 4: K. Chow and Y. Wu. Feedback-directed selection and characterization of compiler optimizations. 2nd Workshop on Feedback Directed Optimization, 1999.
• 5: Compaq. Alpha architecture handbook. whitpaper, October 1998.
• 6: Jeffrey Dean , James E. Hicks , Carl A. Waldspurger , William E. Weihl , George Chrysos, ProfileMe: hardware support for instruction-level profiling on out-of-order processors, Proceedings of the 30th annual ACM/IEEE international symposium on Microarchitecture, p.292-302, December 01-03, 1997, Research Triangle Park, North Carolina, United States
• 7: Jeffrey Dean , James E. Hicks , Carl A. Waldspurger , William E. Weihl , George Chrysos, ProfileMe: hardware support for instruction-level profiling on out-of-order processors, Proceedings of the 30th annual ACM/IEEE international symposium on Microarchitecture, p.292-302, December 01-03, 1997, Research Triangle Park, North Carolina, United States
• 8: Giorgio Delzanno, Automatic Verification of Parameterized Cache Coherence Protocols, Proceedings of the 12th International Conference on Computer Aided Verification, p.53-68, July 15-19, 2000
• 9: Brian A. Fields , Rastislav Bodík , Mark D. Hill , Chris J. Newburn, Using Interaction Costs for Microarchitectural Bottleneck Analysis, Proceedings of the 36th annual IEEE/ACM International Symposium on Microarchitecture, p.228, December 03-05, 2003
• 10: Håkan Grahn , Per Stenström, Evaluation of a competitive-update cache coherence protocol with migratory data detection, Journal of Parallel and Distributed Computing, v.39 n.2, p.168-180, Dec. 15, 1996 [doi>10.1006/jpdc.1996.0164]
• 11: Timothy Heil , James E. Smith, Relational profiling: enabling thread-level parallelism in virtual machines, Proceedings of the 33rd annual ACM/IEEE international symposium on Microarchitecture, p.281-290, December 2000, Monterey, California, United States [doi>10.1145/360128.360156]
• 12: M. Helms, T. Bochner, R. Fritz, T. Schlipf, and M. Walz. Event monitoring in a system-on-a-chip. In Proc. 12th Annual IEEE International ASIC/SOC Conference, Sept. 1999.
• 13: R. Hockauf, J. Jeitner, W. Karl, R. Lindhof, M. Schulz, V. Gonzales, E. Sanquis, and G. Torralba. Design and implementation aspects for the SMiLE hardware monitor. In G. Horn and W. Karl, editors, Proc. of SCI-Europe 2000, The 3rd International Conference on SCI-Based Technology and Research, pages 47--55. SINTEF Electronics and Cybernetics, Aug. 2000. ISBN: 82-595-9964-3, Also available at http://wwwbode.in.tum.de/events/.
• 14: Intel. Intel Itanium Architecture Software Developer's Manual, 2000.
• 15: Intel. Intel Architecture Software Developer's Manual Volume 3: System Programming Guide, 2002.
• 16: Wolfgang Karl , Markus Leberecht , Martin Schulz, Optimizing Data Locality for SCI-Based PC-Clusters with the SMiLE Monitoring Approach, Proceedings of the 1999 International Conference on Parallel Architectures and Compilation Techniques, p.169, October 12-16, 1999
• 17: Margaret Martonosi , Douglas W. Clark , Malena Mesarina, The SHRIMP performance monitor: design and applications, Proceedings of the SIGMETRICS symposium on Parallel and distributed tools, p.61-69, May 22-23, 1996, Philadelphia, Pennsylvania, United States [doi>10.1145/238020.238040]
• 18: Margaret Martonosi , David Ofelt , Mark Heinrich, Integrating performance monitoring and communication in parallel computers, Proceedings of the 1996 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.138-147, May 23-26, 1996, Philadelphia, Pennsylvania, United States
• 19: Tao Mu , Jie Tao , Martin Schulz , Sally A. McKee, Interactive locality optimization on NUMA architectures, Proceedings of the 2003 ACM symposium on Software visualization, June 11-13, 2003, San Diego, California [doi>10.1145/774833.774853]
• 20: Ashwini Nanda , Kwok-Ken Mak , Krishnan Sugavanam , Ramendra K. Sahoo , Vijayaraghavan Soundararajan , T. Basil Smith, MemorIES: a programmable, real-time hardware emulation tool for multiprocessor server design, ACM SIGPLAN Notices, v.35 n.11, p.37-48, Nov. 2000 [doi>10.1145/356989.356993]
• 21: Milos Prvulovic , Josep Torrellas, ReEnact: using thread-level speculation mechanisms to debug data races in multithreaded codes, Proceedings of the 30th annual international symposium on Computer architecture, June 09-11, 2003, San Diego, California
• 22: V. Salapura. Bluegene/p performance counters. Personal Communication: Paper in Submission, Nov. 2007.
• 23: V. Salapura, K. Ganesan, A. Gara, M. Gschwind, J. Sexton, and R. Walkup. Next-generation performance counters: Towards monitoring over thousand concurrent events. Performance Analysis of Systems and software, 2008. ISPASS 2008. IEEE International Symposium on, pages 139--146, April 2008.
• 24: Smruti R. Sarangi , Abhishek Tiwari , Josep Torrellas, Phoenix: Detecting and Recovering from Permanent Processor Design Bugs with Programmable Hardware, Proceedings of the 39th Annual IEEE/ACM International Symposium on Microarchitecture, p.26-37, December 09-13, 2006 [doi>10.1109/MICRO.2006.41]
• 25: S. Subramanya Sastry , Rastislav Bodík , James E. Smith, Rapid profiling via stratified sampling, Proceedings of the 28th annual international symposium on Computer architecture, p.278-289, June 30-July 04, 2001, Göteborg, Sweden
• 26: Martin Schulz , Brian S. White , Sally A. McKee , Hsien-Hsin S. Lee , Jürgen Jeitner, Owl: next generation system monitoring, Proceedings of the 2nd conference on Computing frontiers, May 04-06, 2005, Ischia, Italy [doi>10.1145/1062261.1062284]
• 27: Brinkley Sprunt, The Basics of Performance-Monitoring Hardware, IEEE Micro, v.22 n.4, p.64-71, July 2002 [doi>10.1109/MM.2002.1028477]
• 28: Brinkley Sprunt, Pentium 4 Performance-Monitoring Features, IEEE Micro, v.22 n.4, p.72-82, July 2002 [doi>10.1109/MM.2002.1028478]
• 29: Min Xu , Rastislav Bodik , Mark D. Hill, A "flight data recorder" for enabling full-system multiprocessor deterministic replay, Proceedings of the 30th annual international symposium on Computer architecture, June 09-11, 2003, San Diego, California
• 30: Pin Zhou , Vivek Pandey , Jagadeesan Sundaresan , Anand Raghuraman , Yuanyuan Zhou , Sanjeev Kumar, Dynamic tracking of page miss ratio curve for memory management, Proceedings of the 11th international conference on Architectural support for programming languages and operating systems, October 07-13, 2004, Boston, MA, USA
• 31: Pin Zhou , Feng Qin , Wei Liu , Yuanyuan Zhou , Josep Torrellas, iWatcher: Efficient Architectural Support for Software Debugging, Proceedings of the 31st annual international symposium on Computer architecture, p.224, June 19-23, 2004, München, Germany
• 32: Pin Zhou , Feng Qin , Wei Liu , Yuanyuan Zhou , Josep Torrellas, iWatcher: Efficient Architectural Support for Software Debugging, Proceedings of the 31st annual international symposium on Computer architecture, p.224, June 19-23, 2004, München, Germany

INDEX TERMS

Primary Classification:
  B. Hardware
  B.8 Performance and Reliability
      B.8.2 Performance Analysis and Design Aids

Additional Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.3 Other Architecture Styles

General Terms:
Design

Keywords:
cache coherency, debugging, multicore, performance monitoring, profiling, realtime, scheduling