A case for virtual channel processors

ABSTRACT

Modern desktop and server computer systems use multiple processors: general purpose CPU(s), graphic processor (GPU), network processors (NP) on Network Interface Cards (NICs), RAID controllers, and signal processors on sound cards and modems. Some of these processors traditionally have been special purpose processors but there is a trend towards replacing some of these with embedded general purpose processors. At the same time main CPUs become more powerful; desktop CPUs start featuring Simultaneous Multi-Threading (SMT); and Symmetric Multi-Processing (SMP) systems are widely used in server systems. However, the structure of operating systems has not really changed to reflect these trends --- different types of processors evolve at different timescales (largely driven by market forces) requiring significant changes to operating systems kernels to reflect the appropriate tradeoffs.In this position paper we propose to re-vitalise the old idea of channel processors by encapsulating operating system I/O subsystems in Virtual Channel Processors (VCPs). VCPs perform I/O operations on behalf of an OS. They provide similar development, performance, and fault isolation as dedicated (embedded) I/O processors do while offering the flexibility to split functionality between the main processor(s) and dedicated processors without affecting the rest of the OS. If part of a VCP is executed on the main processor, we propose to make use of virtual machine technology and SMT/SMP features to isolate its performance from that of the rest of the system and to protect the system from faults within the VCP.

REFERENCES

• 1: M. Accetta, R. Baron, W. Bolosky, D. Golub, R. Rashid, A. Tevanian, and M. Young. Mach: a new kernel foundation for UNIX development. In Proc. of Summer Usenix, pages 93--113, July 1986.
• 2: P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, E. Kotsovinos, A. Madhavapeddy, R. Neugebauer, I. Pratt, and A. Warfield. Xen 2002: The Xenoserver Hypervisor. Technical Report UCAM-CL-TR-533, Cambridge Computer Laboratory, Jan. 2003.
• 3: B. N. Bershad , S. Savage , P. Pardyak , E. G. Sirer , M. E. Fiuczynski , D. Becker , C. Chambers , S. Eggers, Extensibility safety and performance in the SPIN operating system, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.267-283, December 03-06, 1995, Copper Mountain, Colorado, United States
• 4: R. J. Black. Explicit Network Scheduling. PhD thesis, University of Cambridge Computer Laboratory, Apr. 1995. Available as Technical Report no. 361.
• 5: José Carlos Brustoloni , Peter Steenkiste, Effects of buffering semantics on I/O performance, Proceedings of the second USENIX symposium on Operating systems design and implementation, p.277-291, October 29-November 01, 1996, Seattle, Washington, United States
• 6: Edouard Bugnion , Scott Devine , Mendel Rosenblum, Disco: running commodity operating systems on scalable multiprocessors, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.143-156, October 05-08, 1997, Saint Malo, France
• 7: J. Chapin , M. Rosenblum , S. Devine , T. Lahiri , D. Teodosiu , A. Gupta, Hive: fault containment for shared-memory multiprocessors, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.12-25, December 03-06, 1995, Copper Mountain, Colorado, United States
• 8: J. Chase, A. Gallatin, and K. Yocum. End-System Optimisation for High-Speed TCP. IEEE Communications, 39(4):68--74, Apr. 2001.
• 9: Andy Chou , Junfeng Yang , Benjamin Chelf , Seth Hallem , Dawson Engler, An empirical study of operating systems errors, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 10: Compaq, Intel, Microsoft. Virtual Interface Architecture Specification, Version 1.0, Dec. 1997.
• 11: R. Cormier, R. Dugan, and R. Guyette. System/370 Extended Architecture: The Channel Subsystem. IBM J. of Research and Development, 27(3):206--218, May 1983.
• 12: F. Hady, T. Bock, M. Cabot, J. Chu, J. Meinecke, K. Oliver, and W. Talarek. Speeding Complex Networking Applications with Twin Cities, a Heterogeneous Multiprocessing Prototype. Accepted for IEEE Network, 2003.
• 13: E. C. Hendricks and T. C. Hartman. Evolution of a Virtual Machine Subsystem. IBM Systems J., 18(1):111--142, 1997.
• 14: IETF. IP Storage Working Group. http://www.ietf.org/html.charters/ips-charter.html.
• 15: R. Illikkal and R. Huggahalli. Benefits of Threads in TCP/IP Processing. Submitted for publication, Jan. 2003.
• 16: Intel Corporation. IA-32 Intel® Architecture Software Developer's Manual; Volume 1: Basic Architecture, 2003.
• 17: Michael B. Jones , Stefan Saroiu, Predictability requirements of a soft modem, Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.37-49, June 2001, Cambridge, Massachusetts, United States
• 18: James R. Larus , Michael Parkes, Using Cohort-Scheduling to Enhance Server Performance, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.103-114, June 10-15, 2002
• 19: J. Liedtke, On micro-kernel construction, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.237-250, December 03-06, 1995, Copper Mountain, Colorado, United States
• 20: Jochen Liedtke, Toward real microkernels, Communications of the ACM, v.39 n.9, p.70-77, Sept. 1996 [doi>10.1145/234215.234473]
• 21: D. MacDonald and W. Barkley. Microsoft Windows 2000 TCP/IP Implementation Details. White paper, Microsoft, 2000.
• 22: D. Marr, F. Binns, D. Hill, G. Hinten, D. Koufaty, J. Miler, and M. Upton. Hyper-Threading Technology Architecture and Microarchitecture. Intel Technical Journal, 6(1), Feb. 2002.
• 23: J. Mogul. TCP offload is a dumb idea whose time has come. In Proc. of the 9th Workshop on Hot Topics in Operating Systems, Lihue, Hawaii, May 2003.
• 24: S. Muir and J. Smith. AsyMOS - An Asymmetric Multiprocessor Operating System. In Proc. of 1st IEEE Conf. on Open Architectures and Network Programming (OPENARCH'98), San Francisco, CA, USA, Apr. 1998.
• 25: Steve Muir , Jonathan Smith, Functional divisions in the Piglet multiprocessor operating system, Proceedings of the 8th ACM SIGOPS European workshop on Support for composing distributed applications, p.255-260, September 1998, Sintra, Portugal [doi>10.1145/319195.319234]
• 26: Vivek S. Pai , Peter Druschel , Willy Zwaenepoel, IO-lite: a unified I/O buffering and caching system, Proceedings of the third symposium on Operating systems design and implementation, p.15-28, February 1999, New Orleans, Louisiana, United States
• 27: I. Pratt and K. Fraser. Arsenic: A User-Accessible Gigabit Ethernet Interface. In Proc. of the 20th Annual Joint Conference of the IEEE Computer and Communications Societies (INFOCOM-01), pages 67--76, Los Alamitos, CA, Apr. 2001.
• 28: M. Rangarajan, A. Bohra, K. Banerjee, E. Carrera, R. Bianchini, L. Iftode, and W. Zwaenepoel. TCP Servers: Offloading TCP/IP Processing in Internet Servers. Design, Implementation, and Performance. Technical Report DCS-TR-481, Rutgers University, Department of Computer Science, Mar. 2002.
• 29: G. Regnier, D. Minturn, G. McAlpine, V. Saletore, and A. Fong. ETA: Experience with a Intel® Xeon™Processor as a Packet Processing Engine. In Proc. of the Symposium on High Performance Interconnects, Palo Alto, CA, Aug. 2003. accepted for publication.
• 30: Prasenjit Sarkar , Sandeep Uttamchandani , Kaladhar Voruganti, Storage Over IP: When Does Hardware Support Help?, Proceedings of the 2nd USENIX Conference on File and Storage Technologies, March 31-31, 2003, San Francisco, CA
• 31: P. Sarkar and K. Voruganti. IP Storage: The Challenge Ahead. In Proc. of the 19th IEEE Symposium on Mass Storage Systems, College Park, MD, USA, Apr. 2002.
• 32: J. Satran, K. Meth, C. Sapuntzakis, M. Chadalapaka, and E. Zeidner. iSCSI. Internet Draft, draft-ietf-ips-iscsi-19.txt, Nov. 2002.
• 33: L. Seawright and R. MacKinnon. VM/370 - A Study of Multiplicity and Usefulness. IBM Systems J., pages 4--17, 1979.
• 34: C. Small and M. Seltzer. VINO: An Integrated Platform for Operating Systems and Database Research. Technical Report TR-30-94, Harvard University, Cambridge, MA, USA, 1994.
• 35: Jeremy Sugerman , Ganesh Venkitachalam , Beng-Hong Lim, Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.1-14, June 25-30, 2001
• 36: Matt Welsh , David Culler , Eric Brewer, SEDA: an architecture for well-conditioned, scalable internet services, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 37: Andrew Whitaker , Marianne Shaw , Steven D. Gribble, Scale and performance in the Denali isolation kernel, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060308]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management

General Terms:
Design

Keywords:
I/O virtualisation, Virtual Channel Processors, protocol offloading