Skip to main content
Springer Nature Link
Log in

Professor Heinz Neudecker and matrix differential calculus

  • Survey Article
  • Published:
Statistical Papers Aims and scope Submit manuscript

Abstract

The late Professor Heinz Neudecker (1933–2017) made significant contributions to the development of matrix differential calculus and its applications to econometrics, psychometrics, statistics, and other areas. In this paper, we present an insightful overview of matrix-oriented findings and their consequential implications in statistics, drawn from a careful selection of works either authored by Professor Neudecker himself or closely aligned with his scientific pursuits. The topics covered include matrix derivatives, vectorisation operators, special matrices, matrix products, inequalities, generalised inverses, moments and asymptotics, and efficiency comparisons within the realm of multivariate linear modelling. Based on the contributions of Professor Neudecker, several results related to matrix derivatives, statistical moments and the multivariate linear model, which can literally be considered to be his top three areas of research enthusiasm, are particularly included.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. We are thankful to Dr. Jan Hauke and Dr. Simo Puntanen for their help to identify the occasion on which Heinz Neudecker and Jerzy K. Baksalary might have possibly met for the first time.

References

  • Abdullah A, Neudecker H, Liu S (1993) Moore–Penrose inverse of a symmetric matrix. Econom Theory 9:703

    Google Scholar 

  • Anderson TW (1963) Asymptotic theory for principal component analysis. Ann Math Stat 34:122–148

    MathSciNet  Google Scholar 

  • Anderson TW (2003) An introduction to multivariate statistical analysis, 3rd edn. Wiley, New York

    Google Scholar 

  • Baksalary JK, Puntanen S (1991) Generalized matrix versions of the Cauchy–Schwarz and Kantorovich inequalities. Aequat. Math. 41:103–110

    MathSciNet  Google Scholar 

  • Baksalary JK, Schipp B, Trenkler G (1992) Some further results on Hermitian matrix inequalities. Linear Algebra Appl. 160:119–129

    MathSciNet  Google Scholar 

  • Baksalary OM, Trenkler G (2010) Core inverse of matrices. Linear Multilinear Algebra 58:681–697

    MathSciNet  Google Scholar 

  • Baksalary OM, Trenkler G (2014) On a generalized core inverse. Appl Math Comput 236:450–457

    MathSciNet  Google Scholar 

  • Baksalary OM, Trenkler G (2021) The Moore-Penrose inverse: a hundred years on a frontline of physics research. Eur Phys J H 46:9

    Google Scholar 

  • Baksalary OM, Trenkler G (2022) An alternative look at the linear regression model. Stat Pap. 63:1499–1509

    MathSciNet  Google Scholar 

  • Bentler PM, Lee S-Y (1978) Matrix derivatives with chain rule and rules for simple, Hadamard, and Kronecker products. J Math Psychol 17:255–262

    MathSciNet  Google Scholar 

  • Bernstein DS (2018) Scalar, vector, and matrix mathematics: theory, facts, and formulas, revised and expanded. Princeton University Press, Princeton

  • Bloomfield P, Watson GG (1975) The inefficiency of least squares. Biometrika 62:121–128

    MathSciNet  Google Scholar 

  • Boswijk HP, Neudecker H (1994) An inequality between perpendicular least-squares and ordinary least-squares. Econom Theory 10:441–442

    Google Scholar 

  • Boswijk HP, Neudecker H, Liu S (1994) A note on the asymptotics of a stochastic vector difference equation. Biometrika 81:216–218

    MathSciNet  Google Scholar 

  • Browne MW (1977) Generalized least squares estimators in the analysis of covariance structures, South African Statistical Journal, 8 (1974) 1–24. Reprinted. In: Aigner DJ, Goldberger AS (eds) Latent variables in socio-economic models. North-Holland, Amsterdam, pp 205–226

    Google Scholar 

  • Browne MW, Shapiro A (1986) The asymptotic covariance matrix of sample correlation coefficients under general conditions. Linear Algebra Appl 82:169–176

    MathSciNet  Google Scholar 

  • Caswell H (2019) Sensitivity analysis: matrix methods in demography and ecology. Springer, Cham

    Google Scholar 

  • Cook RD, Forzani L (2011) On the mean and variance of the generalized inverse of a singular Wishart matrix. Electron J Stat 5:146–158

    MathSciNet  Google Scholar 

  • Drury SW, Liu S, Lu CY, Puntanen S, Styan GPH (2002) Some comments on several matrix inequalities with applications to canonical correlations: historical background and recent developments. Sankhyā A 64:453–507

    MathSciNet  Google Scholar 

  • Eaton ML, Tyler DE (1991) On Wielandt’s inequality and its application to the asymptotic distribution of the eigenvalues of a random symmetric matrix. Ann Stat 19:260–271

    MathSciNet  Google Scholar 

  • Faliva M (1983) Identificazione e stima nel modello lineare ad equazioni simultanee. Vita e Pensiero, Milan

    Google Scholar 

  • Frauendorf E, Neudecker H, Trenkler G (2005) Multivariate data, the arithmetic mean and exchangeability of transformations. Linear Algebra Appl 410:87–95

  • Ghazal GA, Neudecker H (2000) On second-order and fourth-order moments of jointly distributed random matrices: a survey. Linear Algebra Appl 321:61–93

    MathSciNet  Google Scholar 

  • Golden RM (2020) Statistical machine learning: a unified framework. Chapman and Hall/CRC, Boca Raton

    Google Scholar 

  • Heijmans R, Neudecker H (eds) (1987a) The practice of econometrics-studies on demand, forecasting, money and income. International studies in economics and econometrics (ISEE, volume 15). Kluwer Academic Publishers, Dordrecht

  • Heijmans RDH, Neudecker H (1987b) The coefficient of determination revisited. In: Heijmans R, Neudecker H (eds) The practice of econometrics. International studies in economics and econometrics (ISEE, volume 15). Kluwer Academic Publishers, Dordrecht, pp. 191–204

  • Heijmans R, Neudecker H (1998) Estimation of the SURE model. Stat Pap 39:423–430

    MathSciNet  Google Scholar 

  • Heijmans RDH, Pollock DSG, Satorra A (eds) (2000) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht

  • Holgersson T, Pielaszkiewicz J (2020) A collection of moments of the Wishart distribution. In: Holgersson T, Singull M (eds) Recent developments in multivariate and random matrix analysis. Springer, Cham, pp 147–162

    Google Scholar 

  • Imori S, von Rosen D (2020) On the mean and dispersion of the Moore-Penrose generalized inverse of a Wishart matrix. Electron J Linear Algebra 36:124–133

    MathSciNet  Google Scholar 

  • Jochems DB, Neudecker H (1959) Micro-economic business test data compared with traditional statistics. Metrika 2:46–61

    Google Scholar 

  • Khatri CG, Rao CR (1968) Solutions to some functional equations and their applications to characterization of probability distributions. Sankhyā 30:167–180

    MathSciNet  Google Scholar 

  • Khatri CG, Rao CR (1981) Some extensions of the Kantorovich inequality and statistical applications. J Multivar Anal 11:498–505

    MathSciNet  Google Scholar 

  • Khatri CG, Rao CR (1982) Some generalizations of the Kantorovich inequality. Sankhyā Ser A 44:91–102

    MathSciNet  Google Scholar 

  • Knott M (1975) On the minimum efficiency of least squares. Biometrika 62:129–132

    MathSciNet  Google Scholar 

  • Kollo T (1991) Matrix derivative in multivariate statistics. Tartu University Press, Tartu (in Russian)

  • Kollo T (2000) Asymptotic inference based on eigenprojections of covariance and correlation matrices. In: Heijmans RDH, Pollock DSG, Satorra A (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 207–222

    Google Scholar 

  • Kollo T (2008) Multivariate skewness and kurtosis measures with an application in ICA. J Multivar Anal 99:2328–2338

    MathSciNet  Google Scholar 

  • Kollo T, Neudecker H (1993) Asymptotics of eigenvalues and unit length eigenvectors of sample variance and correlation matrices. J Multivar Anal 47:283–300 (Corrigendum in Journal of Multivariate Analysis, 51 (1994) 210)

  • Kollo T, Neudecker H (1996) Asymptotics of eigenvalue-normed eigenvectors of sample variance and correlation matrices. In: Gupta AK, Girko VL (eds.) Multidimensional statistical analysis and theory of random matrices: proceedings of the Sixth Eugene Lukacs Symposium, Bowling Green, 29–30 March 1996, Berlin, Boston: De Gruyter, pp. 111–128

  • Kollo T, Neudecker H (1997a) Asymptotics of Pearson–Hotelling principal component vectors of sample variance and correlation matrices. Behaviormetrika 24:51–69

    Google Scholar 

  • Kollo T, Neudecker H (1997b) The derivative of an orthogonal matrix of eigenvectors of a symmetric matrix. Linear Algebra Appl 264:489–493

    MathSciNet  Google Scholar 

  • Kollo T, von Rosen D (2000) Distribution and density approximation of the covariance matrix in the growth curve model. Statistics 35:1–22

    MathSciNet  Google Scholar 

  • Kollo T, von Rosen D (2005) Advanced multivariate statistics with matrices. Springer, Dordrecht

    Google Scholar 

  • Koning RH, Neudecker H, Wansbeek TJ (1991) Block Kronecker products and the vecb operator. Linear Algebra Appl 149:165–184

    MathSciNet  Google Scholar 

  • Koning RH, Neudecker H, Wansbeek TJ (1992) Unbiased estimation of fourth-order matrix moments. Linear Algebra Appl 160:163–174

    MathSciNet  Google Scholar 

  • Kreijger RG, Neudecker H (1977) Exact linear restrictions on parameters in the general linear model with a singular covariance matrix. J Am Stat Assoc 72:430–432

    MathSciNet  Google Scholar 

  • Liu S (1993) A matrix equality applicable in the analysis of mean-and-covariance structures. Econom Theory 9:691

    Google Scholar 

  • Liu S (1995) Contributions to matrix calculus and applications in econometrics. Thesis Publishers, Amsterdam

    Google Scholar 

  • Liu S (1997) A further matrix version of the Cauchy–Schwarz inequality. Bull Int Linear Algebra Soc 18:32

    Google Scholar 

  • Liu S (1997) Bounds for a ratio of matrix traces. Bull Int Linear Algebra Soc 18:32

    Google Scholar 

  • Liu S (1999) Matrix results on the Khatri–Rao and Tracy–Singh products. Linear Algebra Appl 289:267–277

    MathSciNet  Google Scholar 

  • Liu S (1999) An inequality involving a special Hadamard product. Bull Int Linear Algebra Soc 23:28

    Google Scholar 

  • Liu S (2000) On matrix trace Kantorovich-type inequalities. In: Heijmans RDH, Pollock DSG, Satorra A (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 39–50

    Google Scholar 

  • Liu S (2000) Efficiency comparisons between two estimators based on matrix determinant Kantorovich-type inequalities. Metrika 51:145–155

    MathSciNet  Google Scholar 

  • Liu S (2000) Efficiency comparisons between the OLSE and the BLUE in a singular linear model. J Stat Plan Inference 84:191–200

    MathSciNet  Google Scholar 

  • Liu S (2000) On local influence for elliptical linear models. Stat Pap 41:211–224

    MathSciNet  Google Scholar 

  • Liu S (2002) On Hadamard product of square roots of correlation matrices. Econom Theory 18:1007

    Google Scholar 

  • Liu S (2002) Several inequalities involving Khatri–Rao products of positive semidefinite matrices. Linear Algebra Appl 354:175–186

    MathSciNet  Google Scholar 

  • Liu S (2004) On diagnostics in conditionally heteroskedastic time series models under elliptical distributions. J Appl Prob 41A:393–405

    MathSciNet  Google Scholar 

  • Liu S, Heyde CC (2003) Some efficiency comparisons for estimators from quasi-likelihood and generalized estimating equations. In: Moore M, Froda S, Léger C (eds) Mathematical statistics and applications: festschrift for constance van Eeden. Institute of Mathematical Statistics Lecture Notes-Monograph Series, vol 42. Institute of Mathematical Statistics, USA and Le Centre de Recherches Mathématiques, pp 357–371

  • Liu S, Heyde CC (2008) On estimation in conditional heteroskedastic time series models under non-normal distributions. Stat Pap 49:455–469

    MathSciNet  Google Scholar 

  • Liu S, Heyde CC, Wong WK (2011) Moment matrices in conditional heteroskedastic models under elliptical distributions with applications in AR-ARCH models. Stat Pap 52:621–632

    MathSciNet  Google Scholar 

  • Liu S, King ML (2002) Two Kantorovich-type inequalities and efficiency comparisons between the OLSE and BLUE. J Inequal Appl 7:169–177

    MathSciNet  Google Scholar 

  • Liu S, Leiva V, Zhuang D, Ma T, Figueroa-Zúñiga JI (2022) Matrix differential calculus with applications in the multivariate linear model and its diagnostics. J Multivar Anal 188:104849

    MathSciNet  Google Scholar 

  • Liu S, Lu CY, Puntanen S (2009) Matrix trace Wielandt inequalities with statistical applications. J Stat Plan Inference 139:2254–2260

    MathSciNet  Google Scholar 

  • Liu S, Ma T, Liu Y (2016) Sensitivity analysis in linear models. Spec Matrices 4:225–232

    MathSciNet  Google Scholar 

  • Liu S, Ma T, Polasek W (2014) Spatial system estimators for panel models: a sensitivity and simulation study. Math Comput Simul 101:78–102

    MathSciNet  Google Scholar 

  • Liu S, Neudecker H (1995) Matrix-trace Cauchy–Schwarz inequalities and applications in canonical correlation analysis. Stat Pap 36:287–298

    MathSciNet  Google Scholar 

  • Liu S, Neudecker H (1996) Several matrix Kantorovich-type inequalities. J Math Anal Appl 197:23–26

    MathSciNet  Google Scholar 

  • Liu S, Neudecker H (1997) Kantorovich inequalities and efficiency comparisons for several classes of estimators in linear models. Stat Neerl 51:345–355

    MathSciNet  Google Scholar 

  • Liu S, Neudecker H (1997) Kantorovich and Cauchy–Schwarz inequalities involving two positive semidefinite matrices, and efficiency comparisons for a singular linear model. Linear Algebra Appl 259:209–221

    MathSciNet  Google Scholar 

  • Liu S, Neudecker H (1999) A survey of Cauchy–Schwarz and Kantorovich type matrix inequalities. Stat Pap 40:55–73

    MathSciNet  Google Scholar 

  • Liu S, Neudecker H (2004) Jacobians for the square-root of a positive definite matrix. Bull Int Linear Algebra Soc 32:40

    Google Scholar 

  • Liu S, Neudecker H (2007) Local sensitivity of the restricted least squares estimator in the linear model. Stat Pap 48:525

    Google Scholar 

  • Liu S, Neudecker H (2009) On pseudo maximum likelihood estimation for multivariate time series models with conditional heteroskedasticity. Math Comput Simul 79:2556–2565

    MathSciNet  Google Scholar 

  • Liu S, Polasek W, Neudecker H (1997) Equality conditions for matrix Kantorovich-type inequalities. J Math Anal Appl 212:517–528

    MathSciNet  Google Scholar 

  • Liu S, Polasek W, Sneller R (2012) Sensitivity analysis of SAR estimators: a numerical approximation. J Stat Comput Simul 82:325–342

    MathSciNet  Google Scholar 

  • Liu S, Trenkler G (2008) Hadamard, Khatri–Rao, Kronecker and other matrix products. Int J Inform Syst Sci 4:160–177

    MathSciNet  Google Scholar 

  • Liu S, Trenkler G, Yang ZP (2007) An inequality involving the Khatri–Rao sum of positive definite matrices. Bull Int Linear Algebra Soc 39:32

    Google Scholar 

  • Liu S, Wang H, Liu Y, Liu C (2023a) Matrix derivatives and Kronecker products for the core and generalised core inverses. Presented in abstract at the 29th international workshop on matrices and statistics, IWMS-2023, held at Linköping University, Sweden on 21–23 August 2023

  • Liu Y, Mao C, Leiva V, Liu S, Silva Neto WA (2022) Asymmetric autoregressive models: statistical aspects and a financial application under COVID-19 pandemic. J Appl Stat 49:1323–1347

    MathSciNet  Google Scholar 

  • Liu Y, Mao G, Leiva V, Liu S, Tapia A (2020) Diagnostic analytics for an autoregressive model under the skew-normal distribution. Mathematics 8:693

    Google Scholar 

  • Liu Y, Wang J, Leiva V, Tapia A, Tan W, Liu S (2023b) Robust autoregressive modeling and its diagnostic analytics with a COVID-19 related application. J Appl Stat. https://doi.org/10.1080/02664763.2023.2198178

  • Liu Y, Wang J, Shi D, Leiva V, Liu S (2023c) A score test for detecting extreme values in a vector autoregressive model. J Stat Comput Simul 93:2751–2779

  • Liu Y, Wang J, Yao Z, Liu C, Liu S (2022) Diagnostic analytics for a GARCH model under skew-normal distributions. Commun Stat-Simul Comput. https://doi.org/10.1080/03610918.2022.2157015

  • Lyhagen J (2012) A note on the representation of E[\(x \otimes xx^{\prime }\)] and E[\(x x^{\prime } \otimes x x^{\prime }\)] for the random vector \(x\). Stat Pap 53:697–701

    MathSciNet  Google Scholar 

  • MacRae EC (1974) Matrix derivatives with an application to an adaptive linear decision problem. Ann Stat 2:337–346

    MathSciNet  Google Scholar 

  • Magnus JR (1988) Linear structures. London and Oxford University Press, London

    Google Scholar 

  • Magnus JR (2010) On the concept of matrix derivative. J Multivar Anal 101:2200–2206

    MathSciNet  Google Scholar 

  • Magnus JR, Neudecker H (1979) The commutation matrix: some properties and applications. Ann Stat 7:381–394

    MathSciNet  Google Scholar 

  • Magnus JR, Neudecker H (1980) The elimination matrix: some lemmas and applications. SIAM J Algebr Discrete Methods 1:422–449

    MathSciNet  Google Scholar 

  • Magnus JR, Neudecker H (1985) Matrix differential calculus with applications to simple, Hadamard, and Kronecker products. J Math Psychol 29:474–492

    MathSciNet  Google Scholar 

  • Magnus JR, Neudecker H (1986) Symmetry, 0-1 matrices and Jacobians: a review. Econom Theory 2:157–190

    Google Scholar 

  • Magnu JR, Neudecker H (1988, 1999, 2019) Matrix differential calculus with applications in statistics and econometrics. Wiley, Chichester

  • Martini JWR, Crossa J, Toledo FH, Cuevas J (2020) On Hadamard and Kronecker products in covariance structures for genotype \(\times \) environment interaction. Plant Genome e20033

  • Mathai AM (1997) Jacobians of matrix transformations and functions of matrix argument. World Scientific, Singapore

    Google Scholar 

  • McCulloch CE (1982) Symmetric matrix derivatives with applications. J Am Stat Assoc 77:679–682

    MathSciNet  Google Scholar 

  • Milliken GA, Akdeniz F (1977) A theorem on the difference of the generalized inverses of two nonnegative matrices. Commun Stat Theory Methods 6:73–79

    MathSciNet  Google Scholar 

  • Muirhead RJ (1982) Aspects of multivariate statistical theory. Wiley, New York

    Google Scholar 

  • Nel DG (1980) On matrix differentiation in statistics. S Afr Stat J 14:137–193

    MathSciNet  Google Scholar 

  • Neudecker H (1967a) On matrix procedures for optimizing differentiable scalar functions of matrices. Stat Neerl 21:101–107

    MathSciNet  Google Scholar 

  • Neudecker H (1967b) Matrix methods for econometric research. University of Birmingham. PhD Thesis

  • Neudecker H (1969) Some theorems on matrix differentiation with special reference to Kronecker matrix products. J Am Stat Assoc 64:953–963

    Google Scholar 

  • Neudecker H (1969) A note on Kronecker matrix products and matrix equation systems. SIAM J Appl Math 17:603–606

    MathSciNet  Google Scholar 

  • Neudecker H (1975) A derivation of the Hessian of the (concentrated) likelihood function of the factor model employing the Schur product. Br J Math Stat Psychol 28:152–156

    Google Scholar 

  • Neudecker H (1979) Best linear unbiased estimation of \(\beta \) subject to linear equality constraints in the general linear model. Stat Pap 20:176–182

    MathSciNet  Google Scholar 

  • Neudecker H (1980) A comment on minimization of functions of a positive semidefinite matrix A subject to \(AX = 0^{\prime }\). J Multivar Anal 10:135–139

    MathSciNet  Google Scholar 

  • Neudecker H (1981) On the matrix formulation of Kaiser varimax criterion. Psychometrika 46:343–345

    MathSciNet  Google Scholar 

  • Neudecker H (1982) On two Germane matrix derivatives. Matrix Tensor Q 33:3–12

    MathSciNet  Google Scholar 

  • Neudecker H (1983) On Jacobians of transformations with skew-symmetric, strictly (lower) triangular or diagonal matrix arguments. Linear Multilinear Algebra 14:271–295

    MathSciNet  Google Scholar 

  • Neudecker H (1985a) Recent advances in statistical applications of commutation matrices. In: Grossman W, Pflug G, Vincze I, Wertz W (eds.) Proceedings of the fourth pannonian symposium on mathematical statistics, volume B, Reidel, Dordrecht, pp 239–250

  • Neudecker H (1985) On the dispersion matrix of a matrix quadratic form connected with the noncentral Wishart distribution. Linear Algebra Appl 70:257–262

    MathSciNet  Google Scholar 

  • Neudecker H (1989) A matrix inequality. Econom Theory 5:174–175

    Google Scholar 

  • Neudecker H (1989) A new proof of the Milliken–Akdeniz theorem. Qüestiió 13:81–82

    MathSciNet  Google Scholar 

  • Neudecker H (1990) The variance matrix of a matrix quadratic form under normality assumptions. Statistics 21:455–459

    MathSciNet  Google Scholar 

  • Neudecker H (1992) A matrix trace inequality. J Math Anal Appl 166:302–303

    MathSciNet  Google Scholar 

  • Neudecker H (1995) Mathematical properties of the variance of the multinomial distribution. J Math Anal Appl 189:757–762

    MathSciNet  Google Scholar 

  • Neudecker H (1997) An inequality concerning the Hadamard matrix product. Econom Theory 13:464

    Google Scholar 

  • Neudecker H (2000) On expected values of fourth-degree matrix products of a multinormal matrix variate. In: Kollo T, Tiit E-M, Srivastava M (eds.) New trends in probability and statistics: proceedings of the 6th Tartu conference on multivariate statistics, Tartu, 19–22 August 1999, VSP Utrecht, Netherlands; TEV Vilnius, Lithuania, pp 97–115

  • Neudecker H (2003) On two matrix derivatives by Kollo and von Rosen. Stat Oper Res Trans 27:153–164

    MathSciNet  Google Scholar 

  • Neudecker H (2004) Estimation of the noncentrality matrix of a noncentral Wishart distribution with unit scale matrix: a matrix generalisation of Leung’s domination result. Stat Oper Res Trans 28:191–200

    Google Scholar 

  • Neudecker H (2006) On the asymptotic distribution of the ‘natural’ estimator of Cronbach’s alpha with standardised variates under nonnormality, ellipticity and normality. In: Brown P, Liu S, Sharma D (eds) Contributions to probability and statistics: applications and challenges. World Scientific, Singapore, pp 167–171

    Google Scholar 

  • Neudecker H (2007) A recursion formula for expected negative and positive powers of the central Wishart distribution. Stat Oper Res Trans 31:201–205

    MathSciNet  Google Scholar 

  • Neudecker H (2007) The variance of vec \(S\), where \(S \sim W_m (n, \Omega )\). Stat Pap 48:526

    Google Scholar 

  • Neudecker H (2007) The expected value of \((trAS)SBS\) for \(S \sim W_m (n, \Sigma )\). Stat Pap 48:703

    Google Scholar 

  • Neudecker H (2007) The expected value of \((tr ASBS)S\) for \(S \sim W_m(n, \Sigma )\). Stat Pap 48:703

    Google Scholar 

  • Neudecker H, Liu S (1993) Matrix trace inequalities involving simple Kronecker, and Hadamard products. Econom Theory 9:690

    Google Scholar 

  • Neudecker H, Liu S (1993) Best quadratic and positive semidefinite unbiased estimation of the variance matrix of the multivariate normal distribution. Commun Stat Theory Methods 22:2723–2732

    MathSciNet  Google Scholar 

  • Neudecker H, Liu S (1995) Note on a matrix-concave function. J Math Anal Appl 196:1139–1141

    MathSciNet  Google Scholar 

  • Neudecker H, Liu S (1996) The density of the Moore–Penrose inverse of a random matrix. Linear Algebra Appl 237–238:123–126

    MathSciNet  Google Scholar 

  • Neudecker H, Liu S (2001) Some statistical properties of Hadamard products of random matrices. Stat Pap 42:475–487

    MathSciNet  Google Scholar 

  • Neudecker H, Liu S (2001) Statistical properties of the Hadamard product of random vectors. Stat Pap 42:529–533

    MathSciNet  Google Scholar 

  • Neudecker H, Liu S, Polasek W (1995) The Hadamard product and some of its applications in statistics. Statistics 26:365–373

    MathSciNet  Google Scholar 

  • Neudecker H, Polasek W, Liu S (1995) The heteroskedastic linear regression model and the Hadamard product a note. J Econom 68:361–366

    MathSciNet  Google Scholar 

  • Neudecker H, Satorra A (1991) Simple proof of a general matrix equality. South Afr Stat J 25:79–82

    MathSciNet  Google Scholar 

  • Neudecker H, Satorra A (1993) A Kronecker matrix inequality with a statistical application. Econom Theory 9:524

    Google Scholar 

  • Neudecker H, Satorra A (1996) The algebraic equality of two asymptotic tests for the hypothesis that a normal distribution has a specified correlation matrix. Stat Prob Lett 30:99–103

    MathSciNet  Google Scholar 

  • Neudecker H, Satorra A (2003) On best affine prediction. Stat Pap 44:257–266

    MathSciNet  Google Scholar 

  • Neudecker H, Satorra A, Trenkler G, Liu S (1995) A Kronecker matrix inequality with a statistical application. Econom Theory 11:654–655

    Google Scholar 

  • Neudecker H, Satorra A, van de Velden M (1997) A fundamental matrix result on scaling in multivariate analysis. Econom Theory 13:890

    Google Scholar 

  • Neudecker H, Trenkler G (2002) Third and fourth moment matrices of Vec\(X^{\prime }\) in multivariate analysis. Linear Algebra Appl 354:223–229

    MathSciNet  Google Scholar 

  • Neudecker H, Trenkler G (2005) Estimation of the Kronecker and products of two mean vectors in multivariate analysis. Discuss Math Prob Stat 25:207–215

    MathSciNet  Google Scholar 

  • Neudecker H, Trenkler G (2006) On the approximate variance of a nonlinear function of random variables. In: Brown P, Liu S, Sharma D (eds) Contributions to probability and statistics: applications and challenges. World Scientific, Singapore, pp 172–177

    Google Scholar 

  • Neudecker H, Trenkler G (2006) Estimation of the Hadamard and cross products of two mean vectors in multivariate analysis. Stat Pap 47:481–485

    Google Scholar 

  • Neudecker H, Trenkler G, Liu S (2009) Inverse and determinant of a special symmetric matrix. Stat Pap 50:221

    Google Scholar 

  • Neudecker H, van de Velden M (1998) Two matrix inequalities involving the Moore–Penrose inverse. Econom Theory 14:290–291

    Google Scholar 

  • Neudecker H, van de Velden M (1998) A determinantal identity. Econom Theory 14:525

    Google Scholar 

  • Neudecker H, van de Velden M (2000) A relationship satisfied by two representations of a positive semi-definite matrix. Econom Theory 16:143

    Google Scholar 

  • Neudecker H, Wansbeek T (1983) Some results on commutation matrices with statistical applications. Can J Stat 11:221–231

    MathSciNet  Google Scholar 

  • Neudecker H, Wansbeek T (1987) Fourth-order properties of normally distributed random matrices. Linear Algebra Appl 97:13–21

    MathSciNet  Google Scholar 

  • Neudecker H, Wesselman AM (1990) The asymptotic variance matrix of the sample correlation matrix. Linear Algebra Appl 127:589–599

    MathSciNet  Google Scholar 

  • Neudecker H, Windmeijer FAG (1991) R\(^2\) in seemingly unrelated regression equations. Stat Neerl 45:405–411

  • Neudecker H, Zmyślony R, Trenkler G (2003) Classroom note: estimation of the cross-product of two mean vectors. Int J Math Educ Sci Technol 34:928–935

    Google Scholar 

  • Parring A-M (1979) Calculation of asymptotic parameters of sampling functions. Actu et Comment Univ Tartuensis 492:86–90 (in Russian)

  • Phillips PCB (1992) Generalized inverses of partitioned matrices. Econom Theory 8:426

    Google Scholar 

  • Pollock DSG (1979) The algebra of econometrics. Wiley, Chichester

    Google Scholar 

  • Pollock DSG (1985) Tensor products and matrix differential calculus. Linear Algebra Appl 67:169–193

    MathSciNet  Google Scholar 

  • Pollock DSG (2000) Computers, multilinear algebra and statistics. In: Reijmans RDH, Pollock DSG, Satorra A (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 277–287

    Google Scholar 

  • Pollock DSG (2013) On Kronecker products, tensor products and matrix differential calculus. Int J Comput Math 90:2462–2476

    MathSciNet  Google Scholar 

  • Pollock DSG (2021) Multidimensional arrays, indices and Kronecker products. Econometrics 9:18

    Google Scholar 

  • Przystalski M (2014) Estimation of the covariance matrix in multivariate partially linear models. J Multivar Anal 123:380–385

    MathSciNet  Google Scholar 

  • Puntanen S, Styan GPH, Isotalo J (2011) Matrix tricks for linear statistical models: our personal top twenty. Springer, Berlin

    Google Scholar 

  • Rao CR (1973) Linear statistical inference and its applications, 2nd edn. Wiley, Wiley, New York

    Google Scholar 

  • Rao CR (1985) The inefficiency of least squares: extensions of the Kantorovich inequality. Linear Algebra Appl 70:249–255

    MathSciNet  Google Scholar 

  • Rao CR (2000) Statistical proofs of some matrix inequalities. Linear Algebra Appl 321:307–320

    MathSciNet  Google Scholar 

  • Rao CR (2006) Statistical proofs of some matrix theorems. Int Stat Rev 74:169–185

    Google Scholar 

  • Rao CR (2007) Antieigenvalues and antisingularvalues of a matrix and applications to problems in statistics. Math Inequal Appl 10:471–489

    MathSciNet  Google Scholar 

  • Rao CR, Rao MB (1998) Matrix algebra and its applications to statistics and econometrics. World Scientific, Singapore

    Google Scholar 

  • Satorra A (1992) Asymptotic robust inferences in the analysis of mean and covariance structures. In: Marsden PV (ed) Sociol Methodol. Basil Blackwell, Oxford, pp 249–278

    Google Scholar 

  • Satorra A, Neudecker H (1992) A matrix equality applicable in the analysis of mean-and-covariance structures. Econom Theory 8:581–582

    Google Scholar 

  • Satorra A, Neudecker H (1994) On the asymptotic optimality of alternative minimum-distance estimators in linear latent-variable models. Econom Theory 10:867–883

    MathSciNet  Google Scholar 

  • Satorra A, Neudecker H (2003) A matrix equality useful in goodness-of-fit testing of structural equation models. J Stat Plan Inference 114:63–80

    MathSciNet  Google Scholar 

  • Satorra A, Neudecker H (2015) A theorem on the rank of a product of matrices with illustration of its use in goodness of fit testing. Psychometrika 80:938–948

    MathSciNet  Google Scholar 

  • Seber GAF (2007) A matrix handbook for statisticians. Wiley, New York

    Google Scholar 

  • Schmidt K, Trenkler G (2006) Einführung in die moderne matrix-algebra. Springer, Berlin (in German; first edition in 1998)

  • Schott JR (2016) Matrix analysis for statistics, 3rd edn. Wiley, New York

    Google Scholar 

  • Styan GPH (1973) Hadamard products and multivariate statistical analysis. Linear Algebra Appl 6:217–240

    MathSciNet  Google Scholar 

  • Styan GPH (1983) On some inequalities associated with ordinary least squares and the Kantorovich inequality. In: Huuhtanen P, Liski EP, Pukkila T, Puntanen S (eds) Festschrift for Eino Haikala on his seventieth birthday. Acta Universitatis Tamperensis University of Tampere, Tampere, pp 158–166

    Google Scholar 

  • Tanabe K, Sagae M (1992) An exact Cholesky decomposition and the generalized inverse of the variance-covariance matrix of the multinomial distribution, with Applications. J R Stat Soc Ser B 54:211–219

    MathSciNet  Google Scholar 

  • Theil HH, Neudecker H (1958) Substitution, complementarity, and the residual variation around Engel curves. The review of economic studies 25:114–123; Also in: Raj B, Koerts J (eds.) Henri Theil’s contributions to economics and econometrics. Advanced studies in theoretical and applied econometrics, volume 25. Kluwer Academic Publishers, Dordrecht, 1992, pp 613–626

  • Tracy DS, Singh RP (1972) A new matrix product and its applications in partitioned matrix differentiation. Stat Neerl 26:143–157

    MathSciNet  Google Scholar 

  • Trenkler G (1995) On the singularity of the sample covariance matrix. J Stat Comput Simul 52:172–173

    Google Scholar 

  • Trenkler G (1998) Four square roots of the vector cross product. Math Gaz 82:100–102

    Google Scholar 

  • Trenkler G (2000) On a generalization of the covariance matrix of the multinomial distribution. In: Heijmans DSG, Pollock SA (eds) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Kluwer Academic Publishers, Dordrecht, pp 67–73

  • Trenkler G (2006) On oblique and orthogonal projectors. In: Brown P, Liu S, Sharma D (eds) Contributions to probability and statistics: applications and challenges. World Scientific, Singapore, pp 178–191

    Google Scholar 

  • Trenkler G, Schipp B, Neudecker H, Liu S (1993) Generalized inverses of partitioned matrices. Econom Theory 9:530–533

    Google Scholar 

  • Trenkler G, Puntanen S (2005) A multivariate version of Samuelson’s inequality. Linear Algebra Appl 410:143–149

    MathSciNet  Google Scholar 

  • Turkington DA (2013) Generalized vectorization, cross-products, and matrix calculus. Cambridge University Press, Cambridge

    Google Scholar 

  • Tyler DE (1981) Asymptotic inference for eigenvectors. Ann Stat 9:725–736

    MathSciNet  Google Scholar 

  • van de Velden M (2002) Innovations in multivariate statistical analysis: a festschrift for Heinz Neudecker. Psychometrika 67:317–319

    Google Scholar 

  • van de Velden M, Liu S, Neudecker H (1999) Idempotency of a certain matrix quadratic form. Bull Int Linear Algebra Soc 22:32

    Google Scholar 

  • van de Velden M, Neudecker H (2000) On an eigenvalue property relevant in correspondence analysis and related methods. Linear Algebra Appl 321:347–364

    MathSciNet  Google Scholar 

  • van de Velden M, Satorra A, Neudecker H (1999) The justification of multidimensional scaling under Euclidean conditions. Econom Theory 15:153

    Google Scholar 

  • von Rosen D (1985) Moments and some asymptotic results for maximum likelihood estimates in multivariate linear normal models with special references to the growth curve model. Research Report No. 141, Inst. of Actuar. Mathematics and Math. Statist., University of Stockholm

  • von Rosen D (1988a) Moments for the inverted Wishart distribution. Scand J Stat 15:97–109

  • von Rosen D (1988b) Moments for matrix normal variables. Statistics 19:575–583

    MathSciNet  Google Scholar 

  • von Rosen D (1989) Maximum likelihood estimators in multivariate linear normal models. J Multivar Anal 31:187–200

    MathSciNet  Google Scholar 

  • von Rosen D (1995) Influential observations in multivariate linear models. Scand J Stat 22:207–222

    MathSciNet  Google Scholar 

  • von Rosen D (2018) Bilinear regression analysis. Springer, Cham

    Google Scholar 

  • von Rosen D (2023) Applications of the Kronecker product when solving matrix equations and constructing matrix derivatives. Presented at the 29th International Workshop on Matrices and Statistics, IWMS-2023, held at Linköping University, Sweden on 21–23 August 2023

  • Wang SG, Shao J (1992) Constrained Kantorovich inequalities and relative efficiency of least squares. J Multivar Anal 42:284–298

    MathSciNet  Google Scholar 

  • Wang SG, Yang ZH (2006) Matrix generalised inverse with applications. Beijing University of Technology Press, Beijing (in Chinese)

  • Yanai H, Takeuchi K, Takane Y (2011) Projection matrices, generalized inverse matrices, and singular value decomposition-statistics for social and behavioral sciences. Springer, New York

    Google Scholar 

  • Yang H (1988) Extensions of the Kantorovich inequality and the error ratio efficiency of the mean square. Math Appl 1:85–90 (in Chinese)

  • Yang H, Wang L (2009) An alternative form of the Watson efficiency. J Stat Plan Inference 139:2767–2774

    MathSciNet  Google Scholar 

  • Yang Y (1988) A matrix trace inequality. J Math Anal Appl 133:573–574

    MathSciNet  Google Scholar 

  • Yang ZP, Liu S, Trenkler G (2009) Further inequalities involving the Khatri–Rao product. Linear Algebra Appl 430:2696–2704

    MathSciNet  Google Scholar 

  • Zhang F (2011) Matrix theory: basic results and techniques. Springer, New York

    Google Scholar 

  • Zhang YT (1985) The exact distribution of the Moore–Penrose inverse of X with a density. Multivariate analysis Vl. Elsevier, New York, pp 633–635

    Google Scholar 

  • Zhang Z (2007) Pseudo-inverse multivariate/matrix-variate distributions. J Multivar Anal 98:1684–1692

    MathSciNet  Google Scholar 

Download references

Acknowledgements

We express our gratitude to Professor Heinz Neudecker’s children, Ilia, Hannah and Franz, for providing us with relevant documents and publications, as well as for reading and approving the manuscript.

We also extend our appreciation to the Editors and Reviewers for their constructive comments and insights, which significantly contributed to the improved presentation of the manuscript.

We would like to thank the following individuals for their valuable communications and support over the years: Drs. Peter Boswijk, Richard William Farebrother, Jan Hauke, Emiel Kaper, Augustyn Markiewicz, Stephen Pollock, Simo Puntanen, Willem Saris, Albert Satorra, George Styan, Sven-Oliver Troschke, Michel van de Velden, Tom Wansbeek, and Frank Windmeijer. Special thanks go to Drs. Ejaz Ahmed, Jorge Figueroa-Zúñiga, Stephen Haslett, Xiaogang Huang, Jun Jin, Kazuhiko Kakamu, Takeaki Kariya, Victor Leiva, Hang Liu, Lin Ma, Tiefeng Ma, Gilberto Paula, Jianmei Ren, Ruili Sun, Yoshio Takane, Qijing Yan, Hu Yang, Xiaoping Zhan and Dan Zhuang for their joint efforts and collaborations. We would also like to acknowledge the useful feedback and support received from Drs. Kai-Tai Fang, Chengcheng Hao, Jianhua Hu, Jeffrey Hunter, Lynn Roy LaMotte, Rui Li, Yuli Liang, Hui Liu, Yonghui Liu, Changyu Lu, Seng Huat Ong, Jianxin Pan, Lei Shi, Kunio Shimizu, Martin Singull, Yongge Tian, Kimmo Vehkalahti, Julia Volaufova, Hongxing Wang, Fuzhen Zhang, Shurong Zheng, Fukang Zhu, and other participants during the talk based on partial materials of this manuscript, which was presented at the 27th International Workshop on Matrices and Statistics, IWMS-2019, held at Shanghai University of International Business and Economics, China on 6–9 June 2019.

Finally, we would like to pay tribute to the late Drs. Risto Heijmans, Wolfgang Polasek, and Haruo Yanai, whose dedication to their colleagues and students continues to inspire us. Though they are no longer with us, their legacy lives on.

Funding

Tönu Kollo acknowledges support from the Estonian Research Foundation (Grant Number PRG1197).

Author information

Authors and Affiliations

  1. Faculty of Science and Techonology, University of Canberra, Bruce, Australia

    Shuangzhe Liu

  2. Faculty of Statistics, Dortmund University of Technology, Dortmund, Germany

    Götz Trenkler

  3. Institute of Mathematics and Statistics, University of Tartu, Tartu, Estonia

    Tõnu Kollo

  4. Department of Energy and Technology, Swedish University of Agricultural Sciences, Uppsala, Sweden

    Dietrich von Rosen

  5. Department of Mathematics, Linköping University, Linköping, Sweden

    Dietrich von Rosen

  6. Faculty of Physics, Adam Mickiewicz University, Poznań, Poland

    Oskar Maria Baksalary

Authors
  1. Shuangzhe Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Götz Trenkler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tõnu Kollo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dietrich von Rosen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Oskar Maria Baksalary
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuangzhe Liu.

Ethics declarations

Conflicts of Interest

The authors declare no conflict of interest.

Additional information

This paper is dedicated to the 90th birth anniversary of Professor Heinz Neudecker (1933–2017) with our affection and appreciation for the scientific legacy he left behind.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Trenkler, G., Kollo, T. et al. Professor Heinz Neudecker and matrix differential calculus. Stat Papers 65, 2605–2639 (2024). https://doi.org/10.1007/s00362-023-01499-w

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00362-023-01499-w

Keywords

Search

Navigation