Efficient dispersal of information for security, load balancing, and fault tolerance

ABSTRACT

An Information Dispersal Algorithm (IDA) is developed that breaks a file F of length L = ↿ F↾ into n pieces Fi, l ≤ i ≤ n, each of length ↿Fi↾ = L/m, so that every m pieces suffice for reconstructing F. Dispersal and reconstruction are computationally efficient. The sum of the lengths ↿Fi↾ is (n/m) · L. Since n/m can be chosen to be close to l, the IDA is space efficient. IDA has numerous applications to secure and reliable storage of information in computer networks and even on single disks, to fault-tolerant and efficient transmission of information in networks, and to communications between processors in parallel computers. For the latter problem provably time-efficient and highly fault-tolerant routing on the n-cube is achieved, using just constant size buffers.

REFERENCES

• 1: ASMUTH, C. A., BLAKLEY, G.R. Pooling splitting and restituting information to overcome total failure of some channels of communication. In Proceedings of the 1982 Symposium on Security and Privacy. IEEE Society, New York, 1982, pp. 156-169.
• 2: BERLEKAMP, E.R. Algebraic coding theory. McGraw-Hill, New York, 1968.
• 3: HALL, M. Combinatorial Theory. Wiley, New York, 1980.
• 4: W. Daniel Hillis, The connection machine, MIT Press, Cambridge, MA, 1986
• 5: MIRSKY, L. An Introduction to Linear Algebra. Dover, New York, 1982.
• 6: PIPPENGER, N. Parallel communication with limited buffers. In Proceedings of the IEEE 25th Symposium on Foundations of Computer Science. IEEE, New York, 1984, pp. 127-136.
• 7: RABIN, M.O. Probabilistic algorithms in finite fields. SIAM J. Comput. 9, 1980, 273-280.
• 8: RABIN, i.O. Fingerprinting by random polynomials. Tech. Rep. TR-15-81. Center for Research in Computing Technology. Harvard Univ., Cambridge, Mass., i 98 i.
• 9: RAGHAVAN, P. Probabilistic construction of deterministic algorithms: Approximating packing integer programs. In Proceedings of the IEEE 27th Symposium on Foundations of Computer Science. lt:t:l:., r~ew York, i 986, pp. i 0- i 8.
• 10: Charles L. Seitz, The cosmic cube, Communications of the ACM, v.28 n.1, p.22-33, Jan. 1985 [doi>10.1145/2465.2467]
• 11: Adi Shamir, How to share a secret, Communications of the ACM, v.22 n.11, p.612-613, Nov. 1979 [doi>10.1145/359168.359176]
• 12: llzlilZL, 11. 3. /-~ rnotael Ol )IIVIL; macn}nes anu a companson of various interconnection networks. IEEE Trans. Comput. 28, 12 (1979), 907-917.
• 13: VALIANT, L. G. A scheme for fast parallel communication. SlAM J. Comput. 11, 2 (1982), 3g/-~ -lt-

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.0 General
          Subjects: Security and protection (e.g., firewalls)

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS

  D. Software
  D.4 OPERATING SYSTEMS
      D.4.3 File Systems Management
          Subjects: Distributed file systems
      D.4.5 Reliability
          Subjects: Fault-tolerance

  E. Data
  E.3 DATA ENCRYPTION

  F. Theory of Computation
  F.2 ANALYSIS OF ALGORITHMS AND PROBLEM COMPLEXITY
      F.2.2 Nonnumerical Algorithms and Problems
          Subjects: Routing and layout

General Terms:
Algorithms, Design, Performance, Reliability, Security, Theory