A Procedure for Nonlinear Least Squares Refinement in Adverse Practical Conditions

ABSTRACT

An iterative method is described for computing optimum values of a set of parameters for the best agreement in a least squares between a given set of (experimental) values and corresponding calculated (theoretical) values where these may be dependent non-linearly on the parameters. The method is for use when (a) the amount of work in the calculation of the theoretical values is large as compared with the amount required to solve a set of linear equations in the same number of unknowns; and (b) the amount of work in calculating the first order partial derivatives of all the theoretical values with respect to one parameter would be even greater, so that using finite difference approximations to the partial derivatives is preferable. In addition to setting up and solving the normal equations at each step, the latent vectors of their matrix and estimates of the errors on the current parameter values are computed and used. Also the known physical limits of the range of each parameter are used. The procedure has second order convergence.

REFERENCES

• 1: AITKEN, A. C. Determinants and Matrices, Oliver and Boyd, London, 1948.
• 2: BENSON, F. A., AND BltADSHAW, M.W. Impedance characteristics of glow-discharge tubes in tile frequency range 200 c/s-70 Mc/s. Brit. IRE Symp. at Cambridge, 1964, and The Radio tvrd Electronic Engineer 29 (1965), 313-324.
• 3: BooTIe, G. W., Box, G. E. P., MULLER, M. E., AND PEIRSON, T .I . Forcasting by generalised regression methods, nonlinear estimation. Princeton, IBM Mineo Share program No. 687 WLNL 1, 1959.
• 4: Box, G. E.P. In The Design of Industrial Experiments, Ch. 11. O. L. Davies (Ed.), Oliver ald Boyd, London, 1954.
• 5: ----. Multi-factor designs of first order'. Biometrica 89 (1952), 49-57.
• 6: --. Use of statistical methods in the elucidation of basic mecharfisms. BuU. Inst. Int. Stats. 86, 3 (1957), 215.
• 7: ----. Fitting empirical data. Annals NY Academy of Sciences 86 (1960), 792-816.
• 8: ----. The effect of errors in the factor levels 'and experimental design. Bull. Inst. Int. Stats. 38, 1, 3 (1960), 133, 339.
• 9: -- AND HVNTn, J. S. The 2 k-r fractional factorial designs. Technometrics 3 (1961), 311-351.
• 10: AND HNUNTER, W. G. A useful method for model building. Technometrics 4 (1962), 301-318.
• 11: CcimY, H. B. The method of steepest descent for nonlinear minimisation problems. Quart. Appl. Maths. II (1944), 258-261.
• 12: DAVIDON, W. C. Variable metric method for minimisation. AEC Res. Develop. Rep. ANL 5990 (Rev.), 1959.
• 13: FLETCHER, R., AND POWELL, M. J.D. A rapidly convergent descent method for minimisation. Computer J. 6 (1963), 163.
• 14: -- AND REEVES, C.M. Function minimisation by conjugate gradients. Computer J. 7 (1964), 149.
• 15: HARTLEY, IT. O. The modified Gauss-Newton method for the fitting of nonlinear regression functions by least-squares. Technometrics 3 (1961), 269-280.
• 16: ----. Exact confidence regions for tte parameters in nonlinear regression laws. Biomelrica 51 (1965), 347.
• 17: HASELGROVL C.B. The solution of nonlinear equations and of differential equations with two point boundary conditions. Comput. J. $ (1961), 255.
• 18: HESTENES, M. It., AND STIEFEL, E. Methods of conjugate gradients for solving linear systems. J. Res. NBS 9 (1952), 409.
• 19: HIMSWORTH, F. R.., SPENDLEY, W., AND HZEXT, G. R. The sequential application of simplex designs in optimisation and evolutionary operation. Technometrics 4 (1962), 441.
• 20: KING, P .F . An anMysis of a hydro-electric system. Comput. J. 8 (1960), 161.
• 21: LEVENBERG, 1.. A method for the solution of certain nonlinear problems in least squares. Quart. Appl. Maths. II (1944), 164-168.
• 22: MADDIBON, R.N. Predicting parameters for the optical model analysis of experimental results. Nuclear Physics Computing Group Report 13, Oxford, 1960.
• 23: ----. Optical model analysis of the elastic scattering of low energy deuterons. Proe. Phys. Soe. 79 (1962), 264.
• 24: ----. Optical model analysis of the elastic scattering of 1 MeV neutrons. Nucl. Phys. 55 (1964), 417.
• 25: ----. Nuclear reaction calculations using computer techniques. Doctoral thesis, Oxford, 1964.
• 26: MARQUARDT, D. W., BENNETT, R. G., AND BURR:ELL, E. J. Least squares analysis of electron paramagnetic resonance spectra. J. Molee. Spectroscopy 7 (1961), 269-279.
• 27: ----. An algorithm for least squares estimation of nonlinear parameters. J. Soc. Indust. Appl. Math. 11 (1963), 431-441.
• 28: MARms, D. W., AND TE, G.J. Iterative methods for linear equations with symmetric positive definite matrix. Comput. J. (1961), 242.
• 29: MIRSy, L. An introduction to linear algebra. Clarendon Press, Oxford, 1955.
• 30: PLAcEIT, R. L., &bin BURMAN, J.P. The design of optimum multifaetorial experiments. Biometrika 83 (1946), 305.
• 31: PowmL, M. J. D. An iterative method for finding stationary values of a function of several variables. Comput. J. 5 (1962), 147.
• 32: ----. An efficient method for fiting the minimum of a function of several variables without calculating derivatives. Computer J. 7 (1964), 155.
• 33: -- A method for minimising a sum of squares of nonlinear functions without caleu. latirlg derivatives. Comput. J. 7 (1965), 303-307.
• 34: ROLLETT, J. S. Computing methods in crystallography. Pergamon, New York, 1965.
• 35: ROSENBROCK, H. All automatic method for finding the greatest or least value of a function. Comput. J. 3 (1960), 175.
• 36: TODD, J. The condition of a certain matrix. Proc. Camb. Phil. Soe. 46 (1949), 116-118. (See also, The condition of certain matrices I., Quart. J. Mech. Appl. Math. II (1949), 469-472. Ditto }{I, Arch. Math. 5 (1954), 249-257. Ditto III J. Res. NBS 60 (1958), 1-7.
• 37: VARaA, R. S. Matrix Iterative Analysis. Prentice-Hall, London, 1963.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.6 Optimization
          Subjects: Least squares methods

General Terms:
Design, Experimentation, Performance, Theory