On the role of compression in distributed systems

ABSTRACT

Compression has been used in numerous ways for many years, but recently two factors have combined in a way to push compression to the forefront of distributed systems. First, the disparity between processor speeds and I/O rates is ever-increasing, making it possible to perform compression in software to a much greater extent than was previously feasible. Second, the growth of new applications demanding enormous data rates, such as digital video and audio, makes hardware compression increasingly desirable. I discuss the importance of compression in various environments and describe how compression may be used not only to reduce the demand for disk space, disk bandwidth, and network bandwidth, but also to appear to extend physical memory.

REFERENCES

• 1: Andrew W. Appel , Kai Li, Virtual memory primitives for user programs, Proceedings of the fourth international conference on Architectural support for programming languages and operating systems, p.96-107, April 08-11, 1991, Santa Clara, California, United States
• 2: M. Burrows, C. Jerian, B. Lampson, and T. Mann. On-line data compression in a log-structured file system. Technical Report 85, DEC Systems Research Center, April 1992.
• 3: Vincent Cate , Thomas Gross, Combining the concepts of compression and caching for a two-level filesystem, Proceedings of the fourth international conference on Architectural support for programming languages and operating systems, p.200-211, April 08-11, 1991, Santa Clara, California, United States
• 4: Robert Hagmann. Comments on workstation operating systems and virtual memory. In Proceedings of the Second Workshop on Workstation Operating Systems, pages 43-48, Pacific Grove, CA, September 1989. IEEE.
• 5: John Markoff. Double-hard-disk capacity, through software. The New York Times, February 23 1992.
• 6: Michael N. Nelson , Brent B. Welch , John K. Ousterhout, Caching in the Sprite network file system, ACM Transactions on Computer Systems (TOCS), v.6 n.1, p.134-154, Feb. 1988 [doi>10.1145/35037.42183]
• 7: Mark Nelson, The Data Compression Book, Henry Holt and Co., Inc., New York, NY, 1991
• 8: John K. Ousterhout , Andrew R. Cherenson , Frederick Douglis , Michael N. Nelson , Brent B. Welch, The Sprite Network Operating System, Computer, v.21 n.2, p.23-36, February 1988 [doi>10.1109/2.16]
• 9: Mendel Rosenblum , John K. Ousterhout, The design and implementation of a log-structured file system, ACM Transactions on Computer Systems (TOCS), v.10 n.1, p.26-52, Feb. 1992 [doi>10.1145/146941.146943]
• 10: Mark Taunton. Compressed executables: an exercise in thinking small. In Proceedings of the USENIX 1991 Summer Conference, 1991.
• 11: Ross N. Williams. An extremely fast ZIV-Lempel data compression algorithm. In Data Compression Conference, pages 362-371, April 1991.