Efficient system design space exploration using machine learning techniques

ABSTRACT

Computer manufacturers spend a huge amount of time, resources, and money in designing new systems and newer configurations, and their ability to reduce costs, charge competitive prices and gain market share depends on how good these systems perform. In this work, we develop predictive models for estimating the performance of systems by using performance numbers from only a small fraction of the overall design space. Specifically, we first develop three models, two based on artificial neural networks and another based on linear regression. Using these models, we analyze the published Standard Performance Evaluation Corporation (SPEC) benchmark results and show that by using the performance numbers of only 2% and 5% of the machines in the design space, we can estimate the performance of all the systems within 9.1% and 4.6% on average, respectively. Then, we show that the performance of future systems can be estimated with less than 2.2% error rate on average by using the data of systems from a previous year. We believe that these tools can accelerate the design space exploration significantly and aid in reducing the corresponding research/development cost and time-to-market.

REFERENCES

• 1: Moya F., Moya J. M. and Lopez J. C., Evaluation of Design Space Exploration Strategies. In Proc. of the EUROMICRO Conference, Sep 1999, York, England
• 2: Engin Ïpek , Sally A. McKee , Rich Caruana , Bronis R. de Supinski , Martin Schulz, Efficiently exploring architectural design spaces via predictive modeling, Proceedings of the 12th international conference on Architectural support for programming languages and operating systems, October 21-25, 2006, San Jose, California, USA
• 3: John Cavazos , Christophe Dubach , Felix Agakov , Edwin Bonilla , Michael F. P. O'Boyle , Grigori Fursin , Olivier Temam, Automatic performance model construction for the fast software exploration of new hardware designs, Proceedings of the 2006 international conference on Compilers, architecture and synthesis for embedded systems, October 22-25, 2006, Seoul, Korea [doi>10.1145/1176760.1176765]
• 4: Benjamin C. Lee , David M. Brooks, Accurate and efficient regression modeling for microarchitectural performance and power prediction, Proceedings of the 12th international conference on Architectural support for programming languages and operating systems, October 21-25, 2006, San Jose, California, USA
• 5: The Standard Performance Evaluation Corporation, http://spec.org
• 6: Pang-Ning Tan , Michael Steinbach , Vipin Kumar, Introduction to Data Mining, (First Edition), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2005
• 7: SPSS Clementine version 11, http://www.spss.com/clementine
• 8: Eyerman S., Eeckhout L. and Bosschere K. D. The Shape of the Processor Design Space and its Implications for Early Stage Explorations. In Proc. of Int. Conf. on ACMOS. Mar 2005, Prague, Czech Republic.
• 9: Arijit Ghosh , Tony Givargis, Analytical Design Space Exploration of Caches for Embedded Systems, Proceedings of the conference on Design, Automation and Test in Europe, p.10650, March 03-07, 2003
• 10: Ipek E., de Supinski B. R., Schulz M. and McKee S. A. An Approach to Performance Prediction for Parallel Applications. In Proc. of the Euro-Par. May 2005, Monte de Caparica, Portugal.
• 11: Christophe Dubach , Timothy Jones , Michael O'Boyle, Microarchitectural Design Space Exploration Using an Architecture-Centric Approach, Proceedings of the 40th Annual IEEE/ACM International Symposium on Microarchitecture, p.262-271, December 01-05, 2007 [doi>10.1109/MICRO.2007.26]
• 12: Salman Khan , Polychronis Xekalakis , John Cavazos , Marcelo Cintra, Using PredictiveModeling for Cross-Program Design Space Exploration in Multicore Systems, Proceedings of the 16th International Conference on Parallel Architecture and Compilation Techniques, p.327-338, September 15-19, 2007 [doi>10.1109/PACT.2007.77]

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications

General Terms:
Design, Performance

Keywords:
design space, machine learning, performance prediction