Network performance impact of a lightweight Linux for Cray XT3 compute nodes

ABSTRACT

This poster describes initial performance results comparing a tuned lightweight Linux environment to the standard Catamount lightweight kernel environment on compute nodes of the Cray XT3 system. We have created a lightweight Linux environment that consumes less memory and outperforms Catamount for the selfish micro-benchmark that measures operating system interference. In spite of this, Catamount significantly outperforms our lightweight Linux environment for all network performance micro-benchmarks. Latency and bandwidth performance are more than 20% worse for Linux and 16-byte allreduce performance is 2.5 times worse, even at small numbers of nodes. These results indicate that even a properly configured and tuned Linux environment can still suffer from performance and scalability issues on a highly balanced platform like the XT3. This poster provides a detailed description of our lightweight Linux environment, shows relevant performance results, and describes the important issues that allow Catamount to achieve superior performance.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems

  D. Software
  D.4 OPERATING SYSTEMS
      D.4.0 General

          Nouns: Linux
      D.4.7 Organization and Design
          Subjects: Distributed systems
      D.4.8 Performance
          Subjects: Operational analysis

General Terms:
Design, Performance