Finding race conditions in Erlang with QuickCheck and PULSE

ABSTRACT

We address the problem of testing and debugging concurrent, distributed Erlang applications. In concurrent programs, race conditions are a common class of bugs and are very hard to find in practice. Traditional unit testing is normally unable to help finding all race conditions, because their occurrence depends so much on timing. Therefore, race conditions are often found during system testing, where due to the vast amount of code under test, it is often hard to diagnose the error resulting from race conditions. We present three tools (QuickCheck, PULSE, and a visualizer) that in combination can be used to test and debug concurrent programs in unit testing with a much better possibility of detecting race conditions. We evaluate our method on an industrial concurrent case study and illustrate how we find and analyze the race conditions.

REFERENCES

• 1: Cyrille Artho, Klaus Havelund, and Shinichi Honiden. Visualization of concurrent program executions. In COMPSAC '07: Proc. of the 31st Annual International Computer Software and Applications Conference, pages 541--546, Washington, DC, USA, 2007. IEEE Computer Society.
• 2: Thomas Arts and Lars-Åke Fredlund. Trace analysis of Erlang programs. SIGPLAN Notices, 37 (12): 18--24, 2002.
• 3: Thomas Arts, John Hughes, Joakim Johansson, and Ulf Wiger. Testing Telecoms Software with Quviq QuickCheck. In ERLANG '06: Proc. of the 2006 ACM SIGPLAN workshop on Erlang. ACM, 2006.
• 4: Koen Claessen and John Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP '00: Proc. of the fifth ACM SIGPLAN international conference on Functional programming, pages 268--279, New York, NY, USA, 2000. ACM.
• 5: Mats Cronqvist. Troubleshooting a large Erlang system. In ERLANG '04: Proc. of the 2004 ACM SIGPLAN workshop on Erlang, pages 11--15, New York, NY, USA, 2004. ACM.
• 6: Lars-Åke Fredlund and Hans Svensson. McErlang: a model checker for a distributed functional programming language. SIGPLAN Not., 42 (9): 125--136, 2007.
• 7: Emden R. Gansner and Stephen C. North. An open graph visualization system and its applications. Software - Practice and Experience, 30: 1203--1233, 1999.
• 8: M. P. Herlihy and J. M. Wing. Axioms for concurrent objects. In POPL '87: Proc. of the 14th ACM SIGACT-SIGPLAN symposium on Principles of Prog. Lang., pages 13--26, New York, NY, USA, 1987. ACM.
• 9: John Hughes. QuickCheck Testing for Fun and Profit. In 9th Int. Symp. on Practical Aspects of Declarative Languages. Springer, 2007.
• 10: Dean F. Jerding, John T. Stasko, and Thomas Ball. Visualizing interactions in program executions. In In Proc. of the 19th International Conference on Software Engineering, pages 360--370, 1997.
• 11: Klein, Lu, and Netzer. Detecting race conditions in parallel programs that use semaphores. Algorithmica, 35: 321--345, 2003.
• 12: Leslie Lamport. How to make a multiprocessor computer that correctly executes multiprocess programs. IEEE Transactions on Computers, 28 (9): 690--691, 1979.
• 13: Leslie Lamport. Time, clocks, and the ordering of events in a distributed system. Commun. ACM, 21 (7): 558--565, 1978.
• 14: Shan Lu, Soyeon Park, Eunsoo Seo, and Yuanyuan Zhou. Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. SIGARCH Comput. Archit. News, 36 (1): 329--339, 2008.
• 15: Shahar Maoz, Asaf Kleinbort, and David Harel. Towards trace visualization and exploration for reactive systems. In VLHCC '07: Proc. of the IEEE Symposium on Visual Languages and Human-Centric Computing, pages 153--156, Washington, DC, USA, 2007. IEEE Computer Society.
• 16: Madanlal Musuvathi, Shaz Qadeer, Thomas Ball, Gérard Basler, Piramanayagam Arumuga Nainar, and Iulian Neamtiu. Finding and reproducing heisenbugs in concurrent programs. In OSDI, pages 267--280, 2008.
• 17: Robert H. B. Netzer and Barton P. Miller. On the complexity of event ordering for shared-memory parallel program executions. In In Proc. of the 1990 Int. Conf. on Parallel Processing, pages 93--97, 1990.
• 18: Chang-Seo Park and Koushik Sen. Randomized active atomicity violation detection in concurrent programs. In SIGSOFT '08/FSE-16: Proc. of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, pages 135--145, New York, NY, USA, 2008. ACM.
• 19: Koushik Sen. Race directed random testing of concurrent programs. SIGPLAN Not., 43 (6): 11--21, 2008.
• 20: H. Svensson and L.-Å. Fredlund. A more accurate semantics for distributed Erlang. In Erlang '07: Proc. of the 2007 SIGPLAN Erlang Workshop, pages 43--54, New York, NY, USA, 2007. ACM.
• 21: B. Topol, J.T. Stasko, and V. Sunderam. Integrating visualization support into distributed computing systems. Proc. of the 15th Int. Conf. on: Distributed Computing Systems, pages 19--26, May-Jun 1995.
• 22: Ulf T. Wiger. Extended process registry for Erlang. In ERLANG '07: Proc. of the 2007 SIGPLAN workshop on ERLANG Workshop, pages 1--10, New York, NY, USA, 2007. ACM.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Distributed debugging

General Terms:
Verification

Keywords:
Erlang, QuickCheck, race conditions