Finding more null pointer bugs, but not too many

ABSTRACT

In the summer of 2006, the FindBugs project was challenged to improve the null pointer analysis in FindBugs so that we could find more null pointer bugs. In particular, we were challenged to try to do as well as a publicly available analysis by Reasoning, Inc on version 4.1.24 of Apache Tomcat. Reasoning's report is a result of running their own static analysis tool and using manual auditing to remove false positives. Reasoning reported a total of 9 null pointer warnings in Tomcat 4.1.24, of which only 2 were reported by FindBugs 1.0. While we wanted to improve the analysis in FindBugs, we wanted to retain our current low level of false positives.

As of result of the work presented in this paper, FindBugs now reports 4 of the 9 warnings in Tomcat, shows that one of the warnings reported by Reasoning is a false positive, and classifies the remaining 4 as being dependent on the feasibility of a particular path, which cannot be easier ascertained by a local examination of the source code. Moreover, we found 24 additional null pointer bugs in Tomcat that had been missed by Reasoning, and overall doubled the number of null pointer bugs found by FindBugs while improving the quality and significance of reported defects.

REFERENCES

• 1: Apache Tomcat. http://tomcat.apache.org, 2006.
• 2: Isil Dillig , Thomas Dillig , Alex Aiken, Static error detection using semantic inconsistency inference, Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation, June 10-13, 2007, San Diego, California, USA
• 3: Dawson Engler , Ken Ashcraft, RacerX: effective, static detection of race conditions and deadlocks, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 4: Dawson Engler , David Yu Chen , Seth Hallem , Andy Chou , Benjamin Chelf, Bugs as deviant behavior: a general approach to inferring errors in systems code, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 5: David Evans, Static detection of dynamic memory errors, Proceedings of the ACM SIGPLAN 1996 conference on Programming language design and implementation, p.44-53, May 21-24, 1996, Philadelphia, Pennsylvania, United States
• 6: David Hovemeyer , Jaime Spacco , William Pugh, Evaluating and tuning a static analysis to find null pointer bugs, Proceedings of the 6th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, September 05-06, 2005, Lisbon, Portugal
• 7: W. Pugh. Null pointer detection microbenchmarks. http://findbugs.googlecode.com/svn/trunk/NullPointerBenchmark/, 2006.
• 8: Reasoning, Inc. Reasoning inspection service defect data report for Tomcat, version 4.1.24, January 2003. http://www.reasoning.com/pdf/Tomcat Defect Report.pdf.

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.3 LOGICS AND MEANINGS OF PROGRAMS
      F.3.2 Semantics of Programming Languages
          Subjects: Program analysis

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Reliability

General Terms:
Algorithms, Reliability, Security

Keywords:
FindBugs, Java, bug patterns, bugs, null pointers, software quality, static analysis