|
 ABSTRACT
Modern interconnects like Myrinet and Gigabit Ethernet offer Gb/s speeds which has put the onus of reducing the communication latency on messaging software. This has led to the development of OS bypass protocols which removed the kernel from the critical path and hence reduced the end-to-end latency. With the advent of programmable NICs, many aspects of protocol processing can be offloaded from user space to the NIC leaving the host processor to dedicate more cycles to the application. Many host-offload messaging systems exist for Myrinet; however, nothing similar exits for Gigabit Ethernet. In this paper we propose Ethernet Message Passing (EMP), a completely new zero-copy, OS-bypass messaging layer for Gigabit Ethernet on Alteon NICs where the entire protocol processing is done at the NIC. This messaging system delivers very good performance (latency of 23 us, and throughput of 880 Mb/s). To the best of our knowledge, this is the first NIC-level implementation of a zero-copy message passing layer for Gigabit Ethernet.
REFERENCES
Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.
 |
1
|
Raoul A. F. Bhoedjang , Kees Verstoep , Tim Rühl , Henri E. Bal , Rutger F. H. Hofman, Evaluating design alternatives for reliable communication on high-speed networks, Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, p.71-81, November 2000, Cambridge, Massachusetts, United States
|
| |
2
|
Nanette J. Boden , Danny Cohen , Robert E. Felderman , Alan E. Kulawik , Charles L. Seitz , Jakov N. Seizovic , Wen-King Su, Myrinet: A Gigabit-per-Second Local Area Network, IEEE Micro, v.15 n.1, p.29-36, February 1995
[doi> 10.1109/40.342015]
|
| |
3
|
M. Boosten, R. W. Dobinson, and P. D. V. van der Stok. MESH: Messaging and scheduling for fine-grain parallel processing on commodity platforms. In Proceedings of PDPTA, June 1999.
|
| |
4
|
G. Chiola and G. Ciaccio. GAMMA, http://www.disi.unige.it/project/gamma.
|
| |
5
|
|
| |
6
|
C. Csanady and P. Wyckoff. Bobnet: High-performance message passing for commodity networking components. In Proceedings of PDCN, December 1998.
|
| |
7
|
P. Gilfeather and T. Underwood. Fragmentation and high performance IP. In CAC Workshop, October 2000.
|
| |
8
|
Infiniband. http://www.infinibandta.org.
|
| |
9
|
B. Lawry, R. Wilson, and A. B. Maccabe. OS bypass implementation benchmark. http://www.cs.unm.edu/~maccabe/SSL, 2001.
|
| |
10
|
Richard P. Martin , Amin M. Vahdat , David E. Culler , Thomas E. Anderson, Effects of communication latency, overhead, and bandwidth in a cluster architecture, Proceedings of the 24th annual international symposium on Computer architecture, p.85-97, June 01-04, 1997, Denver, Colorado, United States
|
| |
11
|
MVIA. http://www.nersc.gov/research/FTG/via, 1998.
|
| |
12
|
Netgear. http://www.netgear.com.
|
| |
13
|
S. Pakin, M. Lauria, and A. Chien. High performance messaging on workstations: Illinois Fast Messages (FM) for Myrinet, 1995.
|
| |
14
|
P. Pietikainen. Hardware acceleration of Scheduled Transfer Protocol. http://oss.sgi.com/projects/stp.
|
| |
15
|
I. Pratt and K. Fraser. Arsenic: a user-accessible gigabit ethernet interface. In Proceedings of Infocom, April 2001.
|
| |
16
|
S. Sumimoto, H. Tezuka, A. Hori, H. Harada, T. Takahashi, and Y. Ishikawa. High performance communication using a gigabit ethernet. Technical Report TR-98003, Real World Computing Partnership, 1998.
|
| |
17
|
VI. http://www.viarch.org, 1998.
|
| |
18
|
T. von Eicken , A. Basu , V. Buch , W. Vogels, U-Net: a user-level network interface for parallel and distributed computing (includes URL), Proceedings of the fifteenth ACM symposium on Operating systems principles, p.40-53, December 03-06, 1995, Copper Mountain, Colorado, United States
|
CITED BY 26
|
|
|
|
|
Eitan Frachtenberg , Fabrizio Petrini , Juan Fernandez , Scott Pakin , Salvador Coll, STORM: lightning-fast resource management, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-26, November 16, 2002, Baltimore, Maryland
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Supratik Majumder , Scott Rixner, Comparing Ethernet and Myrinet for MPI communication, Proceedings of the 7th workshop on Workshop on languages, compilers, and run-time support for scalable systems, p.1-7, October 22-23, 2004, Houston, Texas
|
|
|
Jon Beecroft , David Addison , David Hewson , Moray McLaren , Duncan Roweth , Fabrizio Petrini , Jarek Nieplocha, QsNetII: Defining High-Performance Network Design, IEEE Micro, v.25 n.4, p.34-47, July 2005
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Shinji Sumimoto , Kazuichi Ooe , Kouichi Kumon , Taisuke Boku , Mitsuhisa Sato , Akira Ukawa, A scalable communication layer for multi-dimensional hyper crossbar network using multiple gigabit ethernet, Proceedings of the 20th annual international conference on Supercomputing, June 28-July 01, 2006, Cairns, Queensland, Australia
|
|
|
Keiichi Aoki , Hiroki Maruoka , Koichi Wada , Masaaki Ono, Performance evaluation of offloading software modules to cluster network, Proceedings of the 25th conference on Proceedings of the 25th IASTED International Multi-Conference: parallel and distributed computing and networks, p.51-56, February 13-15, 2007, Innsbruck, Austria
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Adobe Acrobat
QuickTime
Windows Media Player
Real Player
Pdf
C.2