Monitoring distributed systems

ABSTRACT

The monitoring of distributed systems involves the collection, interpretation, and display of information concerning the interactions among concurrently executing processes. This information and its display can support the debugging, testing, performance evaluation, and dynamic documentation of distributed systems. General problems associated with monitoring are outlined in this paper, and the architecture of a general purpose, extensible, distributed monitoring system is presented. Three approaches to the display of process interactions are described: textual traces, animated graphical traces, and a combination of aspects of the textual and graphical approaches. The roles that each of these approaches fulfill in monitoring and debugging distributed systems are identified and compared. Monitoring tools for collecting communication statistics, detecting deadlock, controlling the non-deterministic execution of distributed systems, and for using protocol specifications in monitoring are also described. Our discussion is based on experience in the development and use of a monitoring system within a distributed programming environment called Jade. Jade was developed within the Computer Science Department of the University of Calgary and is now being used to support teaching and research at a number of university and research organizations.

REFERENCES

• 1: F. Baiardi , N. De Francesco , E. Matteoli , S. Stefanini , G. Vaglini, Development of a debugger for a concurrent language, ACM SIGSOFT Software Engineering Notes, v.8 n.4, August 1983
• 2: BALZER, R.M. EXDAMS--EXtendable debugging and monitoring system. In Proceedings of AFIPS Spring Joint Computer Conference. AFIPS Press, Reston, Va., 1969, 567-580.
• 3: BATES, P., AND WILEDEN, J.C. An approach to high-level debugging of distributed systems. Softw. Eng. Not. 8, 4 (Aug. 1983), 107.
• 4: BIRTWISTLE, G. M., WYVILL, B. L. M., LEVINSON, D., AND NEAL, R. Visualizing a simulation using animated pictures. In Proceedings of SCS Conference on Simulation in Strongly Typed Languages (San Diego, Calif., Feb. 2-4, 1984). Society for Computer Simulation, San Diego, 1984, 57-61.
• 5: BROWN, M. H., AND SEDGEWICK, R. Techniques for algorithm animation. IEEE Softw. 2, 1 (Jan. 1985), 28.
• 6: David R. Cheriton , Michael A. Malcolm , Lawrence S. Melen , Gary R. Sager, Thoth, a portable real-time operating system, Communications of the ACM, v.22 n.2, p.105-115, Feb. 1979 [doi>10.1145/359060.359074]
• 7: David R. Cheriton , Willy Zwaenepoel, Distributed process groups in the V Kernel, ACM Transactions on Computer Systems (TOCS), v.3 n.2, p.77-107, May 1985 [doi>10.1145/214438.214439]
• 8: Paolo Corsini , Cosimo Antonio Prete, Multibug: Interactive debugging in distributed systems, IEEE Micro, v.6 n.3, p.26-33, June 1986 [doi>10.1109/MM.1986.304671]
• 9: DEWAR, A. A graphical debugger for prolog. Master's thesis, Department of Computer Science, University of Calgary, Calgary, Alberta, Canada. (1985).
• 10: DEWAR, A., AND UNGER, B. Graphical tracing and debugging of simulations. In Proceedings of SCS Conference on Simulation in Strongly Typed Languages (San Diego, Calif., Feb. 2-4, 1984). Society for Computer Simulation, San Diego, 1984, 68-76.
• 11: GARCIA-MOLINA, H., GERMANO, F., AND KOHLER, W.H. Debugging a distributed computing system. IEEE Trans. Softw. Eng. 10, 2 (Mar. 1984), 210.
• 12: HARRISON, M.D. Monitoring a target network to support subsequent host simulation. Res. Rep., Dept. of Computer Science, University of York, Toronto, Ontario, Canada (1984).
• 13: JOYCE, J. J., BIRTWISTLE, G. M., AND WYVILL, B. L.M. ANDES--an environment for animated discrete event simulation. In Proceedings of United Kingdom Simulation Conference (Bath, U.K., May 1984). United Kingdom Simulation Council, 1984.
• 14: JOYCE, J. J., AND UNCER, B.W. Graphical monitoring of distributed systems. In Proceedings of the SCS Conference on AI, Graphics, and Simulation (San Diego, Calif., Jan. 1985). Society for Computer Simulation, San Diego, Calif., 1985, 85-92.
• 15: Leslie Lamport, Time, clocks, and the ordering of events in a distributed system, Communications of the ACM, v.21 n.7, p.558-565, July 1978 [doi>10.1145/359545.359563]
• 16: LOMOW, G. A., AND UNGER, B.W. Distributed software prototyping and simulation in Jade. Can. J. Oper. Res. Inf. Process. 23, 1 (Feb. 1985), 69-89.
• 17: Jeffrey W. Mincy , Alan L. Tharp , Kuo-Chung Tai, Visualizing algorithms and processes with the aid of a computer, Proceedings of the fourteenth SIGCSE technical symposium on Computer science education, p.106-111, February 17-18, 1983, Orlando, Florida, United States
• 18: Mitchell L Model, Monitoring system behavior in a complex computational environment., 1979
• 19: Brad A. Myers, INCENSE: A system for displaying data structures, ACM SIGGRAPH Computer Graphics, v.17 n.3, p.115-125, July 1983
• 20: NEAL, R., LOMOW, G. A., PETERSON, M., UNGER, B. W., AND WITTEN, I.H. Experience with an interprocess communication protocol in a distributed programming environment. In Proceedings of CIPS Session '84 Conference (Calgary, Alberta, Canada, May 9-11, 1984). Canadian Information Processing Society, Calgary, 1984, 361-364.
• 21: NOUNOU, N., AND YEMINI, Y. Development tools for communication protocols. Res. Rep. CUCS-160-85, Department of Computer Science, Columbia University, New York, N.Y. (Feb. 1985).
• 22: K Schwan , J Matthews, Graphical views of parallel programs, ACM SIGSOFT Software Engineering Notes, v.11 n.3, p.51-64, July 1986 [doi>10.1145/12923.12926]
• 23: SNOI)(:~ASS, R.T. Monitoring distributed systems: A relational approach. Ph.D. dissertation, l)epar|ment of Computer Science, Carnegie-Mellon University, Pittsburgh, Pa. (1982).
• 24: SOFTWARE RESEARCH AND DEVEVLOPMENT GROUP. Jade User's Manual (4 vols). Res. Rep., University of Calgary, Dept. of Computer Science, Calgary, Alberta, Canada. (Oct. 1985).
• 25: TEITELMAN, W., AND MASINTER, L. The interlisp programming environment. IEEE Comput. 14, 4 (Apr. 1981).
• 26: UNGER, B. W., AND BIDULOCK, D.S. The design and simulation of a multicomputer network message processor. Comput. Networks 6, 4 (Sept. 1982) 263-277.
• 27: UNGER, B. W., BIRTWISTLE, G. M., CLEARY, J. G., AND DEWAR, A. A distributed software prototyping and simulation environment: Jade. In Proceedings of SCS Conference on Intelligent Simulation Environments (San Diego, Calif., Jan. 23-25, 1986). Society for Computer Simulation, San Diego, 1986, 63-71.
• 28: UNGER, B., CLEARY, J., LOMOW, G., L{, X., SLIND, K., AND XIAO, Z. Jade virtual time implementation manual. Res. Rep. 86/242/16, Department of Computer Science, University of Calgary, Calgary, Alberta, Canada (Oct. 1986).
• 29: VAUCHER, J. Future directions in simulation software. In Proceedings of SCS Conference on Simulation in Strongly Typed Languages (San Diego, Calif., Feb. 1984).
• 30: WYVILL, B. L. M., NEAL, R., LEVINSON, D., AND BRAMWELL, B. JAGGIES: A distributed hierarchical graphics system. In Proceedings of CIPS Session '84 Conference (Calgary, Alberta, May 9-11, 1984). Canadian Information Processing Society, Calgary, 1984, 214-217.

INDEX TERMS

Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications

  D. Software
  D.1 PROGRAMMING TECHNIQUES
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces
      D.2.4 Software/Program Verification
          Subjects: Assertion checkers
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Documentation
  D.4 OPERATING SYSTEMS
      D.4.8 Performance
          Subjects: Monitors

General Terms:
Design, Human Factors, Measurement