Abstract
The late Professor Heinz Neudecker (1933–2017) made significant contributions to the development of matrix differential calculus and its applications to econometrics, psychometrics, statistics, and other areas. In this paper, we present an insightful overview of matrix-oriented findings and their consequential implications in statistics, drawn from a careful selection of works either authored by Professor Neudecker himself or closely aligned with his scientific pursuits. The topics covered include matrix derivatives, vectorisation operators, special matrices, matrix products, inequalities, generalised inverses, moments and asymptotics, and efficiency comparisons within the realm of multivariate linear modelling. Based on the contributions of Professor Neudecker, several results related to matrix derivatives, statistical moments and the multivariate linear model, which can literally be considered to be his top three areas of research enthusiasm, are particularly included.
Similar content being viewed by others
Notes
We are thankful to Dr. Jan Hauke and Dr. Simo Puntanen for their help to identify the occasion on which Heinz Neudecker and Jerzy K. Baksalary might have possibly met for the first time.
References
Abdullah A, Neudecker H, Liu S (1993) Moore–Penrose inverse of a symmetric matrix. Econom Theory 9:703
Anderson TW (1963) Asymptotic theory for principal component analysis. Ann Math Stat 34:122–148
Anderson TW (2003) An introduction to multivariate statistical analysis, 3rd edn. Wiley, New York
Baksalary JK, Puntanen S (1991) Generalized matrix versions of the Cauchy–Schwarz and Kantorovich inequalities. Aequat. Math. 41:103–110
Baksalary JK, Schipp B, Trenkler G (1992) Some further results on Hermitian matrix inequalities. Linear Algebra Appl. 160:119–129
Baksalary OM, Trenkler G (2010) Core inverse of matrices. Linear Multilinear Algebra 58:681–697
Baksalary OM, Trenkler G (2014) On a generalized core inverse. Appl Math Comput 236:450–457
Baksalary OM, Trenkler G (2021) The Moore-Penrose inverse: a hundred years on a frontline of physics research. Eur Phys J H 46:9
Baksalary OM, Trenkler G (2022) An alternative look at the linear regression model. Stat Pap. 63:1499–1509
Bentler PM, Lee S-Y (1978) Matrix derivatives with chain rule and rules for simple, Hadamard, and Kronecker products. J Math Psychol 17:255–262
Bernstein DS (2018) Scalar, vector, and matrix mathematics: theory, facts, and formulas, revised and expanded. Princeton University Press, Princeton
Bloomfield P, Watson GG (1975) The inefficiency of least squares. Biometrika 62:121–128
Boswijk HP, Neudecker H (1994) An inequality between perpendicular least-squares and ordinary least-squares. Econom Theory 10:441–442
Boswijk HP, Neudecker H, Liu S (1994) A note on the asymptotics of a stochastic vector difference equation. Biometrika 81:216–218
Browne MW (1977) Generalized least squares estimators in the analysis of covariance structures, South African Statistical Journal, 8 (1974) 1–24. Reprinted. In: Aigner DJ, Goldberger AS (eds) Latent variables in socio-economic models. North-Holland, Amsterdam, pp 205–226
Browne MW, Shapiro A (1986) The asymptotic covariance matrix of sample correlation coefficients under general conditions. Linear Algebra Appl 82:169–176
Caswell H (2019) Sensitivity analysis: matrix methods in demography and ecology. Springer, Cham
Cook RD, Forzani L (2011) On the mean and variance of the generalized inverse of a singular Wishart matrix. Electron J Stat 5:146–158
Drury SW, Liu S, Lu CY, Puntanen S, Styan GPH (2002) Some comments on several matrix inequalities with applications to canonical correlations: historical background and recent developments. Sankhyā A 64:453–507
Eaton ML, Tyler DE (1991) On Wielandt’s inequality and its application to the asymptotic distribution of the eigenvalues of a random symmetric matrix. Ann Stat 19:260–271
Faliva M (1983) Identificazione e stima nel modello lineare ad equazioni simultanee. Vita e Pensiero, Milan
Frauendorf E, Neudecker H, Trenkler G (2005) Multivariate data, the arithmetic mean and exchangeability of transformations. Linear Algebra Appl 410:87–95
Ghazal GA, Neudecker H (2000) On second-order and fourth-order moments of jointly distributed random matrices: a survey. Linear Algebra Appl 321:61–93
Golden RM (2020) Statistical machine learning: a unified framework. Chapman and Hall/CRC, Boca Raton
Heijmans R, Neudecker H (eds) (1987a) The practice of econometrics-studies on demand, forecasting, money and income. International studies in economics and econometrics (ISEE, volume 15). Kluwer Academic Publishers, Dordrecht
Heijmans RDH, Neudecker H (1987b) The coefficient of determination revisited. In: Heijmans R, Neudecker H (eds) The practice of econometrics. International studies in economics and econometrics (ISEE, volume 15). Kluwer Academic Publishers, Dordrecht, pp. 191–204
Heijmans R, Neudecker H (1998) Estimation of the SURE model. Stat Pap 39:423–430
Heijmans RDH, Pollock DSG, Satorra A (eds) (2000) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht
Holgersson T, Pielaszkiewicz J (2020) A collection of moments of the Wishart distribution. In: Holgersson T, Singull M (eds) Recent developments in multivariate and random matrix analysis. Springer, Cham, pp 147–162
Imori S, von Rosen D (2020) On the mean and dispersion of the Moore-Penrose generalized inverse of a Wishart matrix. Electron J Linear Algebra 36:124–133
Jochems DB, Neudecker H (1959) Micro-economic business test data compared with traditional statistics. Metrika 2:46–61
Khatri CG, Rao CR (1968) Solutions to some functional equations and their applications to characterization of probability distributions. Sankhyā 30:167–180
Khatri CG, Rao CR (1981) Some extensions of the Kantorovich inequality and statistical applications. J Multivar Anal 11:498–505
Khatri CG, Rao CR (1982) Some generalizations of the Kantorovich inequality. Sankhyā Ser A 44:91–102
Knott M (1975) On the minimum efficiency of least squares. Biometrika 62:129–132
Kollo T (1991) Matrix derivative in multivariate statistics. Tartu University Press, Tartu (in Russian)
Kollo T (2000) Asymptotic inference based on eigenprojections of covariance and correlation matrices. In: Heijmans RDH, Pollock DSG, Satorra A (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 207–222
Kollo T (2008) Multivariate skewness and kurtosis measures with an application in ICA. J Multivar Anal 99:2328–2338
Kollo T, Neudecker H (1993) Asymptotics of eigenvalues and unit length eigenvectors of sample variance and correlation matrices. J Multivar Anal 47:283–300 (Corrigendum in Journal of Multivariate Analysis, 51 (1994) 210)
Kollo T, Neudecker H (1996) Asymptotics of eigenvalue-normed eigenvectors of sample variance and correlation matrices. In: Gupta AK, Girko VL (eds.) Multidimensional statistical analysis and theory of random matrices: proceedings of the Sixth Eugene Lukacs Symposium, Bowling Green, 29–30 March 1996, Berlin, Boston: De Gruyter, pp. 111–128
Kollo T, Neudecker H (1997a) Asymptotics of Pearson–Hotelling principal component vectors of sample variance and correlation matrices. Behaviormetrika 24:51–69
Kollo T, Neudecker H (1997b) The derivative of an orthogonal matrix of eigenvectors of a symmetric matrix. Linear Algebra Appl 264:489–493
Kollo T, von Rosen D (2000) Distribution and density approximation of the covariance matrix in the growth curve model. Statistics 35:1–22
Kollo T, von Rosen D (2005) Advanced multivariate statistics with matrices. Springer, Dordrecht
Koning RH, Neudecker H, Wansbeek TJ (1991) Block Kronecker products and the vecb operator. Linear Algebra Appl 149:165–184
Koning RH, Neudecker H, Wansbeek TJ (1992) Unbiased estimation of fourth-order matrix moments. Linear Algebra Appl 160:163–174
Kreijger RG, Neudecker H (1977) Exact linear restrictions on parameters in the general linear model with a singular covariance matrix. J Am Stat Assoc 72:430–432
Liu S (1993) A matrix equality applicable in the analysis of mean-and-covariance structures. Econom Theory 9:691
Liu S (1995) Contributions to matrix calculus and applications in econometrics. Thesis Publishers, Amsterdam
Liu S (1997) A further matrix version of the Cauchy–Schwarz inequality. Bull Int Linear Algebra Soc 18:32
Liu S (1997) Bounds for a ratio of matrix traces. Bull Int Linear Algebra Soc 18:32
Liu S (1999) Matrix results on the Khatri–Rao and Tracy–Singh products. Linear Algebra Appl 289:267–277
Liu S (1999) An inequality involving a special Hadamard product. Bull Int Linear Algebra Soc 23:28
Liu S (2000) On matrix trace Kantorovich-type inequalities. In: Heijmans RDH, Pollock DSG, Satorra A (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 39–50
Liu S (2000) Efficiency comparisons between two estimators based on matrix determinant Kantorovich-type inequalities. Metrika 51:145–155
Liu S (2000) Efficiency comparisons between the OLSE and the BLUE in a singular linear model. J Stat Plan Inference 84:191–200
Liu S (2000) On local influence for elliptical linear models. Stat Pap 41:211–224
Liu S (2002) On Hadamard product of square roots of correlation matrices. Econom Theory 18:1007
Liu S (2002) Several inequalities involving Khatri–Rao products of positive semidefinite matrices. Linear Algebra Appl 354:175–186
Liu S (2004) On diagnostics in conditionally heteroskedastic time series models under elliptical distributions. J Appl Prob 41A:393–405
Liu S, Heyde CC (2003) Some efficiency comparisons for estimators from quasi-likelihood and generalized estimating equations. In: Moore M, Froda S, Léger C (eds) Mathematical statistics and applications: festschrift for constance van Eeden. Institute of Mathematical Statistics Lecture Notes-Monograph Series, vol 42. Institute of Mathematical Statistics, USA and Le Centre de Recherches Mathématiques, pp 357–371
Liu S, Heyde CC (2008) On estimation in conditional heteroskedastic time series models under non-normal distributions. Stat Pap 49:455–469
Liu S, Heyde CC, Wong WK (2011) Moment matrices in conditional heteroskedastic models under elliptical distributions with applications in AR-ARCH models. Stat Pap 52:621–632
Liu S, King ML (2002) Two Kantorovich-type inequalities and efficiency comparisons between the OLSE and BLUE. J Inequal Appl 7:169–177
Liu S, Leiva V, Zhuang D, Ma T, Figueroa-Zúñiga JI (2022) Matrix differential calculus with applications in the multivariate linear model and its diagnostics. J Multivar Anal 188:104849
Liu S, Lu CY, Puntanen S (2009) Matrix trace Wielandt inequalities with statistical applications. J Stat Plan Inference 139:2254–2260
Liu S, Ma T, Liu Y (2016) Sensitivity analysis in linear models. Spec Matrices 4:225–232
Liu S, Ma T, Polasek W (2014) Spatial system estimators for panel models: a sensitivity and simulation study. Math Comput Simul 101:78–102
Liu S, Neudecker H (1995) Matrix-trace Cauchy–Schwarz inequalities and applications in canonical correlation analysis. Stat Pap 36:287–298
Liu S, Neudecker H (1996) Several matrix Kantorovich-type inequalities. J Math Anal Appl 197:23–26
Liu S, Neudecker H (1997) Kantorovich inequalities and efficiency comparisons for several classes of estimators in linear models. Stat Neerl 51:345–355
Liu S, Neudecker H (1997) Kantorovich and Cauchy–Schwarz inequalities involving two positive semidefinite matrices, and efficiency comparisons for a singular linear model. Linear Algebra Appl 259:209–221
Liu S, Neudecker H (1999) A survey of Cauchy–Schwarz and Kantorovich type matrix inequalities. Stat Pap 40:55–73
Liu S, Neudecker H (2004) Jacobians for the square-root of a positive definite matrix. Bull Int Linear Algebra Soc 32:40
Liu S, Neudecker H (2007) Local sensitivity of the restricted least squares estimator in the linear model. Stat Pap 48:525
Liu S, Neudecker H (2009) On pseudo maximum likelihood estimation for multivariate time series models with conditional heteroskedasticity. Math Comput Simul 79:2556–2565
Liu S, Polasek W, Neudecker H (1997) Equality conditions for matrix Kantorovich-type inequalities. J Math Anal Appl 212:517–528
Liu S, Polasek W, Sneller R (2012) Sensitivity analysis of SAR estimators: a numerical approximation. J Stat Comput Simul 82:325–342
Liu S, Trenkler G (2008) Hadamard, Khatri–Rao, Kronecker and other matrix products. Int J Inform Syst Sci 4:160–177
Liu S, Trenkler G, Yang ZP (2007) An inequality involving the Khatri–Rao sum of positive definite matrices. Bull Int Linear Algebra Soc 39:32
Liu S, Wang H, Liu Y, Liu C (2023a) Matrix derivatives and Kronecker products for the core and generalised core inverses. Presented in abstract at the 29th international workshop on matrices and statistics, IWMS-2023, held at Linköping University, Sweden on 21–23 August 2023
Liu Y, Mao C, Leiva V, Liu S, Silva Neto WA (2022) Asymmetric autoregressive models: statistical aspects and a financial application under COVID-19 pandemic. J Appl Stat 49:1323–1347
Liu Y, Mao G, Leiva V, Liu S, Tapia A (2020) Diagnostic analytics for an autoregressive model under the skew-normal distribution. Mathematics 8:693
Liu Y, Wang J, Leiva V, Tapia A, Tan W, Liu S (2023b) Robust autoregressive modeling and its diagnostic analytics with a COVID-19 related application. J Appl Stat. https://doi.org/10.1080/02664763.2023.2198178
Liu Y, Wang J, Shi D, Leiva V, Liu S (2023c) A score test for detecting extreme values in a vector autoregressive model. J Stat Comput Simul 93:2751–2779
Liu Y, Wang J, Yao Z, Liu C, Liu S (2022) Diagnostic analytics for a GARCH model under skew-normal distributions. Commun Stat-Simul Comput. https://doi.org/10.1080/03610918.2022.2157015
Lyhagen J (2012) A note on the representation of E[\(x \otimes xx^{\prime }\)] and E[\(x x^{\prime } \otimes x x^{\prime }\)] for the random vector \(x\). Stat Pap 53:697–701
MacRae EC (1974) Matrix derivatives with an application to an adaptive linear decision problem. Ann Stat 2:337–346
Magnus JR (1988) Linear structures. London and Oxford University Press, London
Magnus JR (2010) On the concept of matrix derivative. J Multivar Anal 101:2200–2206
Magnus JR, Neudecker H (1979) The commutation matrix: some properties and applications. Ann Stat 7:381–394
Magnus JR, Neudecker H (1980) The elimination matrix: some lemmas and applications. SIAM J Algebr Discrete Methods 1:422–449
Magnus JR, Neudecker H (1985) Matrix differential calculus with applications to simple, Hadamard, and Kronecker products. J Math Psychol 29:474–492
Magnus JR, Neudecker H (1986) Symmetry, 0-1 matrices and Jacobians: a review. Econom Theory 2:157–190
Magnu JR, Neudecker H (1988, 1999, 2019) Matrix differential calculus with applications in statistics and econometrics. Wiley, Chichester
Martini JWR, Crossa J, Toledo FH, Cuevas J (2020) On Hadamard and Kronecker products in covariance structures for genotype \(\times \) environment interaction. Plant Genome e20033
Mathai AM (1997) Jacobians of matrix transformations and functions of matrix argument. World Scientific, Singapore
McCulloch CE (1982) Symmetric matrix derivatives with applications. J Am Stat Assoc 77:679–682
Milliken GA, Akdeniz F (1977) A theorem on the difference of the generalized inverses of two nonnegative matrices. Commun Stat Theory Methods 6:73–79
Muirhead RJ (1982) Aspects of multivariate statistical theory. Wiley, New York
Nel DG (1980) On matrix differentiation in statistics. S Afr Stat J 14:137–193
Neudecker H (1967a) On matrix procedures for optimizing differentiable scalar functions of matrices. Stat Neerl 21:101–107
Neudecker H (1967b) Matrix methods for econometric research. University of Birmingham. PhD Thesis
Neudecker H (1969) Some theorems on matrix differentiation with special reference to Kronecker matrix products. J Am Stat Assoc 64:953–963
Neudecker H (1969) A note on Kronecker matrix products and matrix equation systems. SIAM J Appl Math 17:603–606
Neudecker H (1975) A derivation of the Hessian of the (concentrated) likelihood function of the factor model employing the Schur product. Br J Math Stat Psychol 28:152–156
Neudecker H (1979) Best linear unbiased estimation of \(\beta \) subject to linear equality constraints in the general linear model. Stat Pap 20:176–182
Neudecker H (1980) A comment on minimization of functions of a positive semidefinite matrix A subject to \(AX = 0^{\prime }\). J Multivar Anal 10:135–139
Neudecker H (1981) On the matrix formulation of Kaiser varimax criterion. Psychometrika 46:343–345
Neudecker H (1982) On two Germane matrix derivatives. Matrix Tensor Q 33:3–12
Neudecker H (1983) On Jacobians of transformations with skew-symmetric, strictly (lower) triangular or diagonal matrix arguments. Linear Multilinear Algebra 14:271–295
Neudecker H (1985a) Recent advances in statistical applications of commutation matrices. In: Grossman W, Pflug G, Vincze I, Wertz W (eds.) Proceedings of the fourth pannonian symposium on mathematical statistics, volume B, Reidel, Dordrecht, pp 239–250
Neudecker H (1985) On the dispersion matrix of a matrix quadratic form connected with the noncentral Wishart distribution. Linear Algebra Appl 70:257–262
Neudecker H (1989) A matrix inequality. Econom Theory 5:174–175
Neudecker H (1989) A new proof of the Milliken–Akdeniz theorem. Qüestiió 13:81–82
Neudecker H (1990) The variance matrix of a matrix quadratic form under normality assumptions. Statistics 21:455–459
Neudecker H (1992) A matrix trace inequality. J Math Anal Appl 166:302–303
Neudecker H (1995) Mathematical properties of the variance of the multinomial distribution. J Math Anal Appl 189:757–762
Neudecker H (1997) An inequality concerning the Hadamard matrix product. Econom Theory 13:464
Neudecker H (2000) On expected values of fourth-degree matrix products of a multinormal matrix variate. In: Kollo T, Tiit E-M, Srivastava M (eds.) New trends in probability and statistics: proceedings of the 6th Tartu conference on multivariate statistics, Tartu, 19–22 August 1999, VSP Utrecht, Netherlands; TEV Vilnius, Lithuania, pp 97–115
Neudecker H (2003) On two matrix derivatives by Kollo and von Rosen. Stat Oper Res Trans 27:153–164
Neudecker H (2004) Estimation of the noncentrality matrix of a noncentral Wishart distribution with unit scale matrix: a matrix generalisation of Leung’s domination result. Stat Oper Res Trans 28:191–200
Neudecker H (2006) On the asymptotic distribution of the ‘natural’ estimator of Cronbach’s alpha with standardised variates under nonnormality, ellipticity and normality. In: Brown P, Liu S, Sharma D (eds) Contributions to probability and statistics: applications and challenges. World Scientific, Singapore, pp 167–171
Neudecker H (2007) A recursion formula for expected negative and positive powers of the central Wishart distribution. Stat Oper Res Trans 31:201–205
Neudecker H (2007) The variance of vec \(S\), where \(S \sim W_m (n, \Omega )\). Stat Pap 48:526
Neudecker H (2007) The expected value of \((trAS)SBS\) for \(S \sim W_m (n, \Sigma )\). Stat Pap 48:703
Neudecker H (2007) The expected value of \((tr ASBS)S\) for \(S \sim W_m(n, \Sigma )\). Stat Pap 48:703
Neudecker H, Liu S (1993) Matrix trace inequalities involving simple Kronecker, and Hadamard products. Econom Theory 9:690
Neudecker H, Liu S (1993) Best quadratic and positive semidefinite unbiased estimation of the variance matrix of the multivariate normal distribution. Commun Stat Theory Methods 22:2723–2732
Neudecker H, Liu S (1995) Note on a matrix-concave function. J Math Anal Appl 196:1139–1141
Neudecker H, Liu S (1996) The density of the Moore–Penrose inverse of a random matrix. Linear Algebra Appl 237–238:123–126
Neudecker H, Liu S (2001) Some statistical properties of Hadamard products of random matrices. Stat Pap 42:475–487
Neudecker H, Liu S (2001) Statistical properties of the Hadamard product of random vectors. Stat Pap 42:529–533
Neudecker H, Liu S, Polasek W (1995) The Hadamard product and some of its applications in statistics. Statistics 26:365–373
Neudecker H, Polasek W, Liu S (1995) The heteroskedastic linear regression model and the Hadamard product a note. J Econom 68:361–366
Neudecker H, Satorra A (1991) Simple proof of a general matrix equality. South Afr Stat J 25:79–82
Neudecker H, Satorra A (1993) A Kronecker matrix inequality with a statistical application. Econom Theory 9:524
Neudecker H, Satorra A (1996) The algebraic equality of two asymptotic tests for the hypothesis that a normal distribution has a specified correlation matrix. Stat Prob Lett 30:99–103
Neudecker H, Satorra A (2003) On best affine prediction. Stat Pap 44:257–266
Neudecker H, Satorra A, Trenkler G, Liu S (1995) A Kronecker matrix inequality with a statistical application. Econom Theory 11:654–655
Neudecker H, Satorra A, van de Velden M (1997) A fundamental matrix result on scaling in multivariate analysis. Econom Theory 13:890
Neudecker H, Trenkler G (2002) Third and fourth moment matrices of Vec\(X^{\prime }\) in multivariate analysis. Linear Algebra Appl 354:223–229
Neudecker H, Trenkler G (2005) Estimation of the Kronecker and products of two mean vectors in multivariate analysis. Discuss Math Prob Stat 25:207–215
Neudecker H, Trenkler G (2006) On the approximate variance of a nonlinear function of random variables. In: Brown P, Liu S, Sharma D (eds) Contributions to probability and statistics: applications and challenges. World Scientific, Singapore, pp 172–177
Neudecker H, Trenkler G (2006) Estimation of the Hadamard and cross products of two mean vectors in multivariate analysis. Stat Pap 47:481–485
Neudecker H, Trenkler G, Liu S (2009) Inverse and determinant of a special symmetric matrix. Stat Pap 50:221
Neudecker H, van de Velden M (1998) Two matrix inequalities involving the Moore–Penrose inverse. Econom Theory 14:290–291
Neudecker H, van de Velden M (1998) A determinantal identity. Econom Theory 14:525
Neudecker H, van de Velden M (2000) A relationship satisfied by two representations of a positive semi-definite matrix. Econom Theory 16:143
Neudecker H, Wansbeek T (1983) Some results on commutation matrices with statistical applications. Can J Stat 11:221–231
Neudecker H, Wansbeek T (1987) Fourth-order properties of normally distributed random matrices. Linear Algebra Appl 97:13–21
Neudecker H, Wesselman AM (1990) The asymptotic variance matrix of the sample correlation matrix. Linear Algebra Appl 127:589–599
Neudecker H, Windmeijer FAG (1991) R\(^2\) in seemingly unrelated regression equations. Stat Neerl 45:405–411
Neudecker H, Zmyślony R, Trenkler G (2003) Classroom note: estimation of the cross-product of two mean vectors. Int J Math Educ Sci Technol 34:928–935
Parring A-M (1979) Calculation of asymptotic parameters of sampling functions. Actu et Comment Univ Tartuensis 492:86–90 (in Russian)
Phillips PCB (1992) Generalized inverses of partitioned matrices. Econom Theory 8:426
Pollock DSG (1979) The algebra of econometrics. Wiley, Chichester
Pollock DSG (1985) Tensor products and matrix differential calculus. Linear Algebra Appl 67:169–193
Pollock DSG (2000) Computers, multilinear algebra and statistics. In: Reijmans RDH, Pollock DSG, Satorra A (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 277–287
Pollock DSG (2013) On Kronecker products, tensor products and matrix differential calculus. Int J Comput Math 90:2462–2476
Pollock DSG (2021) Multidimensional arrays, indices and Kronecker products. Econometrics 9:18
Przystalski M (2014) Estimation of the covariance matrix in multivariate partially linear models. J Multivar Anal 123:380–385
Puntanen S, Styan GPH, Isotalo J (2011) Matrix tricks for linear statistical models: our personal top twenty. Springer, Berlin
Rao CR (1973) Linear statistical inference and its applications, 2nd edn. Wiley, Wiley, New York
Rao CR (1985) The inefficiency of least squares: extensions of the Kantorovich inequality. Linear Algebra Appl 70:249–255
Rao CR (2000) Statistical proofs of some matrix inequalities. Linear Algebra Appl 321:307–320
Rao CR (2006) Statistical proofs of some matrix theorems. Int Stat Rev 74:169–185
Rao CR (2007) Antieigenvalues and antisingularvalues of a matrix and applications to problems in statistics. Math Inequal Appl 10:471–489
Rao CR, Rao MB (1998) Matrix algebra and its applications to statistics and econometrics. World Scientific, Singapore
Satorra A (1992) Asymptotic robust inferences in the analysis of mean and covariance structures. In: Marsden PV (ed) Sociol Methodol. Basil Blackwell, Oxford, pp 249–278
Satorra A, Neudecker H (1992) A matrix equality applicable in the analysis of mean-and-covariance structures. Econom Theory 8:581–582
Satorra A, Neudecker H (1994) On the asymptotic optimality of alternative minimum-distance estimators in linear latent-variable models. Econom Theory 10:867–883
Satorra A, Neudecker H (2003) A matrix equality useful in goodness-of-fit testing of structural equation models. J Stat Plan Inference 114:63–80
Satorra A, Neudecker H (2015) A theorem on the rank of a product of matrices with illustration of its use in goodness of fit testing. Psychometrika 80:938–948
Seber GAF (2007) A matrix handbook for statisticians. Wiley, New York
Schmidt K, Trenkler G (2006) Einführung in die moderne matrix-algebra. Springer, Berlin (in German; first edition in 1998)
Schott JR (2016) Matrix analysis for statistics, 3rd edn. Wiley, New York
Styan GPH (1973) Hadamard products and multivariate statistical analysis. Linear Algebra Appl 6:217–240
Styan GPH (1983) On some inequalities associated with ordinary least squares and the Kantorovich inequality. In: Huuhtanen P, Liski EP, Pukkila T, Puntanen S (eds) Festschrift for Eino Haikala on his seventieth birthday. Acta Universitatis Tamperensis University of Tampere, Tampere, pp 158–166
Tanabe K, Sagae M (1992) An exact Cholesky decomposition and the generalized inverse of the variance-covariance matrix of the multinomial distribution, with Applications. J R Stat Soc Ser B 54:211–219
Theil HH, Neudecker H (1958) Substitution, complementarity, and the residual variation around Engel curves. The review of economic studies 25:114–123; Also in: Raj B, Koerts J (eds.) Henri Theil’s contributions to economics and econometrics. Advanced studies in theoretical and applied econometrics, volume 25. Kluwer Academic Publishers, Dordrecht, 1992, pp 613–626
Tracy DS, Singh RP (1972) A new matrix product and its applications in partitioned matrix differentiation. Stat Neerl 26:143–157
Trenkler G (1995) On the singularity of the sample covariance matrix. J Stat Comput Simul 52:172–173
Trenkler G (1998) Four square roots of the vector cross product. Math Gaz 82:100–102
Trenkler G (2000) On a generalization of the covariance matrix of the multinomial distribution. In: Heijmans DSG, Pollock SA (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 67–73
Trenkler G (2006) On oblique and orthogonal projectors. In: Brown P, Liu S, Sharma D (eds) Contributions to probability and statistics: applications and challenges. World Scientific, Singapore, pp 178–191
Trenkler G, Schipp B, Neudecker H, Liu S (1993) Generalized inverses of partitioned matrices. Econom Theory 9:530–533
Trenkler G, Puntanen S (2005) A multivariate version of Samuelson’s inequality. Linear Algebra Appl 410:143–149
Turkington DA (2013) Generalized vectorization, cross-products, and matrix calculus. Cambridge University Press, Cambridge
Tyler DE (1981) Asymptotic inference for eigenvectors. Ann Stat 9:725–736
van de Velden M (2002) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Psychometrika 67:317–319
van de Velden M, Liu S, Neudecker H (1999) Idempotency of a certain matrix quadratic form. Bull Int Linear Algebra Soc 22:32
van de Velden M, Neudecker H (2000) On an eigenvalue property relevant in correspondence analysis and related methods. Linear Algebra Appl 321:347–364
van de Velden M, Satorra A, Neudecker H (1999) The justification of multidimensional scaling under Euclidean conditions. Econom Theory 15:153
von Rosen D (1985) Moments and some asymptotic results for maximum likelihood estimates in multivariate linear normal models with special references to the growth curve model. Research Report No. 141, Inst. of Actuar. Mathematics and Math. Statist., University of Stockholm
von Rosen D (1988a) Moments for the inverted Wishart distribution. Scand J Stat 15:97–109
von Rosen D (1988b) Moments for matrix normal variables. Statistics 19:575–583
von Rosen D (1989) Maximum likelihood estimators in multivariate linear normal models. J Multivar Anal 31:187–200
von Rosen D (1995) Influential observations in multivariate linear models. Scand J Stat 22:207–222
von Rosen D (2018) Bilinear regression analysis. Springer, Cham
von Rosen D (2023) Applications of the Kronecker product when solving matrix equations and constructing matrix derivatives. Presented at the 29th International Workshop on Matrices and Statistics, IWMS-2023, held at Linköping University, Sweden on 21–23 August 2023
Wang SG, Shao J (1992) Constrained Kantorovich inequalities and relative efficiency of least squares. J Multivar Anal 42:284–298
Wang SG, Yang ZH (2006) Matrix generalised inverse with applications. Beijing University of Technology Press, Beijing (in Chinese)
Yanai H, Takeuchi K, Takane Y (2011) Projection matrices, generalized inverse matrices, and singular value decomposition-statistics for social and behavioral sciences. Springer, New York
Yang H (1988) Extensions of the Kantorovich inequality and the error ratio efficiency of the mean square. Math Appl 1:85–90 (in Chinese)
Yang H, Wang L (2009) An alternative form of the Watson efficiency. J Stat Plan Inference 139:2767–2774
Yang Y (1988) A matrix trace inequality. J Math Anal Appl 133:573–574
Yang ZP, Liu S, Trenkler G (2009) Further inequalities involving the Khatri–Rao product. Linear Algebra Appl 430:2696–2704
Zhang F (2011) Matrix theory: basic results and techniques. Springer, New York
Zhang YT (1985) The exact distribution of the Moore–Penrose inverse of X with a density. Multivariate analysis Vl. Elsevier, New York, pp 633–635
Zhang Z (2007) Pseudo-inverse multivariate/matrix-variate distributions. J Multivar Anal 98:1684–1692
Acknowledgements
We express our gratitude to Professor Heinz Neudecker’s children, Ilia, Hannah and Franz, for providing us with relevant documents and publications, as well as for reading and approving the manuscript.
We also extend our appreciation to the Editors and Reviewers for their constructive comments and insights, which significantly contributed to the improved presentation of the manuscript.
We would like to thank the following individuals for their valuable communications and support over the years: Drs. Peter Boswijk, Richard William Farebrother, Jan Hauke, Emiel Kaper, Augustyn Markiewicz, Stephen Pollock, Simo Puntanen, Willem Saris, Albert Satorra, George Styan, Sven-Oliver Troschke, Michel van de Velden, Tom Wansbeek, and Frank Windmeijer. Special thanks go to Drs. Ejaz Ahmed, Jorge Figueroa-Zúñiga, Stephen Haslett, Xiaogang Huang, Jun Jin, Kazuhiko Kakamu, Takeaki Kariya, Victor Leiva, Hang Liu, Lin Ma, Tiefeng Ma, Gilberto Paula, Jianmei Ren, Ruili Sun, Yoshio Takane, Qijing Yan, Hu Yang, Xiaoping Zhan and Dan Zhuang for their joint efforts and collaborations. We would also like to acknowledge the useful feedback and support received from Drs. Kai-Tai Fang, Chengcheng Hao, Jianhua Hu, Jeffrey Hunter, Lynn Roy LaMotte, Rui Li, Yuli Liang, Hui Liu, Yonghui Liu, Changyu Lu, Seng Huat Ong, Jianxin Pan, Lei Shi, Kunio Shimizu, Martin Singull, Yongge Tian, Kimmo Vehkalahti, Julia Volaufova, Hongxing Wang, Fuzhen Zhang, Shurong Zheng, Fukang Zhu, and other participants during the talk based on partial materials of this manuscript, which was presented at the 27th International Workshop on Matrices and Statistics, IWMS-2019, held at Shanghai University of International Business and Economics, China on 6–9 June 2019.
Finally, we would like to pay tribute to the late Drs. Risto Heijmans, Wolfgang Polasek, and Haruo Yanai, whose dedication to their colleagues and students continues to inspire us. Though they are no longer with us, their legacy lives on.
Funding
Tönu Kollo acknowledges support from the Estonian Research Foundation (Grant Number PRG1197).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
The authors declare no conflict of interest.
Additional information
This paper is dedicated to the 90th birth anniversary of Professor Heinz Neudecker (1933–2017) with our affection and appreciation for the scientific legacy he left behind.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liu, S., Trenkler, G., Kollo, T. et al. Professor Heinz Neudecker and matrix differential calculus. Stat Papers 65, 2605–2639 (2024). https://doi.org/10.1007/s00362-023-01499-w
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00362-023-01499-w