Algorithm 717: Subroutines for maximum likelihood and quasi-likelihood estimation of parameters in nonlinear regression models

ABSTRACT

We present FORTRAN 77 subroutines that solve statistical parameter estimation problems for general nonlinear models, e.g., nonlinear least-squares, maximum likelihood, maximum quasi-likelihood, generalized nonlinear least-squares, and some robust fitting problems. The accompanying test examples include members of the generalized linear model family, extensions using nonlinear predictors (“nonlinear GLIM”), and probabilistic choice models, such as linear-in-parameter multinomial probit models. The basic method, a generalization of the NL2SOL algorithm for nonlinear least-squares, employs a model/trust-region scheme for computing trial steps, exploits special structure by maintaining a secant approximation to the second-order part of the Hessian, and adaptively switches between a Gauss-Newton and an augmented Hessian approximation. Gauss-Newton steps are computed using a corrected seminormal equations approach. The subroutines include variants that handle simple bounds on the parameters, and that compute approximate regression diagnostics.

REFERENCES

• 1: The PORT Mathematical Subroutine Library, 3rd ed. AT & T Bell Laboratories, Murray Hill, NJ, 1984.
• 2: BARD, Y. Nonlinear Parameter Estimatton. Academic Press, 1974.
• 3: BELSLEY, D. A., KUH, E., AND WELSCH, R.E. Regression Dtagnostics: Identifying Influential Data and Sources of Collineartty. Wiley, New York, 1980.
• 4: BJ~RCK, h. Stability analysis of the method of seminormal equations for least squares problems. Linear Algebra Appl. 88/89 (Apr. 1987), 31 48.
• 5: David S. Bunch, Maximum likelihood estimation of probabilistic choice methods, SIAM Journal on Scientific and Statistical Computing, v.8 n.1, p.56-70, January 2, 1987 [doi>10.1137/0908006]
• 6: BUNCH, D. S. A comparison of algorithms for maximum likelihood estimation of choice models. J. Econometrtcs 38 (1988), 145-167.
• 7: BUNCH, D. S. A comparison of algorithms for maximum likelihood estimation of finite distribution models. Working Paper UCD-GSM-WP1590, Graduate School of Management, Univ. of California, Davis, CA, 1989.
• 8: BUNCH, D.S. MLMNP and MLMNPB: Fortran programs for maximum likelihood estimation of linear-in-parameter multinomial probit models. Rep. UCD-ITS-RR-91-14, Inst. of Transportation Studies, Univ. of California, Davis, CA, 1991.
• 9: BUNCH, D. S., AND KITAMU~, R. Multinomial probit estimation revisited: Testing estimable model specifications, maximum likelihood algorithms, and probit integral approximations for trinomial models of household car ownership. Rep. UCD-TRG-RR-4, Transportation Research Group, Univ. of California, Davis, CA, 1990.
• 10: R. J. Carroll , D. Ruppert, Transformation and weighting in regression, Chapman & Hall, Ltd., London, UK, 1988
• 11: Samprit Chatterjee, Sensitivity analysis in linear regression, John Wiley & Sons, Inc., New York, NY, 1988
• 12: COOK, R. D., AND WEISBERG, S. Residuals and Influence in Regression. Chapman and Hall, London, 1982.
• 13: DAaANzo, C. Multtnomial Probit: The Theory and Its Application to Demand Forecasting. Academic Press, New York, 1979.
• 14: DAWDtAN, M., AND CARROLL, R.J. Variance function estimation. J. Am. Stat. Assoc. 82,400 (Dec. 1987), 1079-1091.
• 15: John E. Dennis, Jr. , David M. Gay , Roy E. Walsh, An Adaptive Nonlinear Least-Squares Algorithm, ACM Transactions on Mathematical Software (TOMS), v.7 n.3, p.348-368, Sept. 1981 [doi>10.1145/355958.355965]
• 16: John E. Dennis, Jr. , David M. Gay , Roy E. Welsch, Algorithm 573: NL2SOL—An Adaptive Nonlinear Least-Squares Algorithm [E4], ACM Transactions on Mathematical Software (TOMS), v.7 n.3, p.369-383, Sept. 1981 [doi>10.1145/355958.355966]
• 17: DENNIS, J. E., JR., AND WALKER, H.F. Convergence theorems for least-change secant update methods SlAM J. Numer. Anal. 18, 6 (Dec 1981), 949-987.
• 18: DENNIS, J E., JR., AND WALKER, H. F. Erratum: Convergence theorems for least-change secant update methods. SIAM J. Numer. Anal. 19, 2 (Apr. 1982), 443.
• 19: P. A. Fox , A. P. Hall , N. L. Schryer, The PORT Mathematical Subroutine Library, ACM Transactions on Mathematical Software (TOMS), v.4 n.2, p.104-126, June 1978 [doi>10.1145/355780.355783]
• 20: FROME, E.L. Regression methods for binomial and Poisson distributed data. Manuscript, Mathematics and Statistics Research Section, Oak Ridge Natmnal Laboratory, Oak Ridge, TN, 1984. Presented at AAPM Fzrst Mzdyear Topzcal Symposium on Multiple Regression Analysis: Applications in the Health Sciences (Mobfie, AL, Mar. 12 16, 1984).
• 21: GAY, D.M. Computing optimal locally constrained steps SIAM J. Scl. Stat. Comput. 2, 2 (June 1981), 186 197.
• 22: David M. Gay, Algorithm 611: Subroutines for Unconstrained Minimization Using a Model/Trust-Region Approach, ACM Transactions on Mathematical Software (TOMS), v.9 n.4, p.503-524, Dec. 1983 [doi>10.1145/356056.356066]
• 23: GAY, D.M. A trust-region approach to linearly constrained optimization. Numerical Analysis Proceedz~lgs (Dundee, 1983) D. F. Griffiths, Ed, Springer-Verlag, 72-105.
• 24: GAY, D. M. Usage summary for selected optimization routines. Computing Science Tech. Rep. No. 153 (1990), AT & T Bell Laboratories, Murray Hill, NJ. Postscript for this report is avafiable by e-maih ask netlib(~,research.att.com to send 153 from research/cstr.
• 25: GAY, D M., AND WELSCH, R. E. Maximum likelihood and quasi-likehhood for nonlinear exponential family regression models. J. Am. Stat Assoc. 83, 404 (Dec. 1988), 990 998.
• 26: HA~IPEL, F. R., RONCnETTI, E. M., ROUSSEEUW, P. J., AND STAHEL, W.A. Robust Statzst~cs. Wiley, 1986.
• 27: HOLLAND, P. W., AND WELSCH, R.E. Robust regression using ~teratively reweighted leastsquares. Commun. Stat. A6, 9 (1977), 813 827.
• 28: HUBER, P.J. Robust StatLstlcs. Wiley, New York, 1981.
• 29: McCULLAGH, P., AND NELDER, J.A. Generahzed Lznear Models. Chapman and Hall, 1983.
• 30: McCULLAGH, P.~ AND NELDER, J.A. Generalized Ltnear Models, 2nd ed. Chapman and Hall, 1989.
• 31: Mog~, J.J. The Levenberg-Marquardt algorithm: Implementation and theory. In Numerical Analysis, Dundee 1977, G. A. Watson, Ed., Sprmger-Verlag, Berlim 1978, 105-116.
• 32: MOR~, J. J., AND SORENSEN, D. C. Computing a trust region step. SIAM J. Sc~. Stat. Comput. 4, 3 (Sept. 1983), 553-572.
• 33: NELDER, J. A., AND PREGIBON, D. An extended quasi-likelihood function. Blornetr~ka 74, 2 (Aug. 1987), 221 232.
• 34: WALKER, H. F, AND GONGLEWSK~, J D. Quasi-Newton methods for maximum-hkelihood estimation. In Advances in Numerzcal Partial Dzfferent~al Equations and Optimization, S. Gdmez, J. P. Hennart and R. A. Tapia, Eds., SIAM, 1991, 332-345. In Proceedings of the Fifth Mexico-United States Workshop.
• 35: WEDDERBURN, R. W. M. Quasi-likelihood functions, generahzed linear models, and the Gauss-Newton method. Biometr~ka 61, 3 (Dec. 1974), 439 447.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical computing

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical software