A routine for converting regression algorithms into corresponding orthogonal regression algorithms

ABSTRACT

The routine converts any standard regression algorithm (that calculates both the coefficients and residuals) into a corresponding orthogonal regression algorithm. Thus, a standard, or robust, or L1 regression algorithm is converted into the corresponding standard, or robust, or L1 orthogonal algorithm. Such orthogonal procedures are important for three basic reasons. First, they solve the classical errors-in-variables (EV) regression problem. Standard L2 orthogonal regression, obtained by converting ordinary least squares regression, is the maximum likelihood solution of the EV problem under Gaussian assumptions. However, this L2 solution is known to be unstable under even slight deviations from the model. Thus this routine's ability to create robust orthogonal regression algorithms from robust ordinary regression algorithms will also be very useful in practice. Second, orthogonal regression is intimately related to principal components procedures. Therefore, this routine can also be used to create corresponding L1, robust, etc., principal components algorithms. And third, orthogonal regression treats the x and y variables symmetrically. This is very important in many science and engineering modeling problems. Monte Carlo studies, which test the effectiveness of the routine under a variety of types of data, are given.

REFERENCES

• 1: BATES, D., AND DRAPER, N. Applied regression analysis bibliography update 1981-82. Tech. Rep., Dept. of Statistics, Univ. of Wisconsin, Madison, 1983.
• 2: Labs Bell , John M. Chambers , Peter J. Bickel , William S. Cleveland , Richard M. Dudley, S an Interactive Environment for Data Analysis and Graphics, CRC Press, Inc., Boca Raton, FL, 1984
• 3: BROWN, M. Robust line estimation with errors in both variables. J. Am. Star. Assoc. 77, 377 (1982), 71-79.
• 4: CHENG, C. L., AND VAN NESS, J. Robust errors-in-variables regression. Tech. Rep., Mathematical Sciences Program, Univ. of Texas at Dallas, Richardson, Tex., 1987.
• 5: David Coleman , Paul Holland , Neil Kaden , Virginia Klema , Stephen C. Peters, A System of Subroutines for Iteratively Reweighted Least Squares Computations, ACM Transactions on Mathematical Software (TOMS), v.6 n.3, p.327-336, Sept. 1980 [doi>10.1145/355900.355905]
• 6: GNANADESIKAN, R. Methods for Statistical Data Analysis of Multivariate Observations. Wiley, New York, 1977.
• 7: KELLY, G. The influence function in the errors in variables problem. Ann. Star. 12, i (1984), 87-100.
• 8: KENDALL, M., AND STUART, A. The Advanced Theory of Statistics. Vol. 2, 2nd ed., Hafner, New York, 1967.
• 9: LAKSHMINARAYANAN, M. Y., AND GUNST, R.F. Estimation of parameters in linear structural relationships: sensitivity to the choice of the ratio of error variances. Biometrika 71, 3 (1984), 569-574.
• 10: LI, G., AND CHEN, Z. Projection-pursuit approach to robust dispersion matrices and principal components: primary theory and Monte Carlo. J. Am. Star. Assoc. 80, 391 (1985), 759-766.
• 11: LONGLE~, J.W. An appraisal of least-squares programs from the point of view of the user. J. Am. Star. Assoc. 62 (1967), 819-841.
• 12: MALINVAUD, 1~. ~tat~st~cal Metnoas o! ~conometr~cs. zna ea., INorl~ll-rlollan(l, Amsteraam, I~TU.
• 13: MORRXSON, D. Multivariate Statistical Methods. 2nd ed., McGraw-Hill, New York, 1976.
• 14: MUIRHEAD, R. Aspects of Multivariate Statistical Theory. Wiley, New York, 1982.
• 15: RABINOWICZ, E. An Introduction to Experimentation. Addison-Wesley, Reading, Mass., 1970.
• 16: WOLTER, K. M., AND FULLER, W.A. Estimation of the quadratic errors-in-variables model. Biometrika 69, 1 (1982), 175-182.

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing

Additional Classification:
  G. Mathematics of Computing

  J. Computer Applications

General Terms:
Algorithms, Reliability