Energy/performance modeling for collective communication in 3-D torus cluster networks

ABSTRACT

As supercomputers scale ever larger, energy consumption in interconnection networks is an emerging problem. In this work, we analyze the energy consumption and traffic patterns in a 3-D torus network in order to locate and exploit opportunities to save energy by disabling network links dynamically. Using a custom-built simulator, TorusSim, we show that, for common scientific computing codes that utilize collective communications, regularities in the network and algorithmic data flow result in many unused and under-utilized links that can be disabled to save energy at no performance cost. In the case of a reduce operation, we see that at least 56% of the links in a 4x4x4 torus network can be disabled during communication, with significant other opportunity to save energy on under-utilized links that could lead to over 80% overall link energy savings.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.0 GENERAL
      Subjects: Modeling of computer architecture

Additional Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.2 Multiple Data Stream Architectures (Multiprocessors)
          Subjects: Interconnection architectures (e.g., common bus, multiport memory, crossbar switch)
  C.4 PERFORMANCE OF SYSTEMS
  C.5 COMPUTER SYSTEM IMPLEMENTATION
      C.5.1 Large and Medium ("Mainframe") Computers
          Subjects: Super (very large) computers

General Terms:
Design, Performance