Abstract
Factor analysis refers to a statistical model in which observed variables are conditionally independent given fewer hidden variables, known as factors, and all the random variables follow a multivariate normal distribution. The parameter space of a factor analysis model is a subset of the cone of positive definite matrices. This parameter space is studied from the perspective of computational algebraic geometry. Gröbner bases and resultants are applied to compute the ideal of all polynomial functions that vanish on the parameter space. These polynomials, known as model invariants, arise from rank conditions on a symmetric matrix under elimination of the diagonal entries of the matrix. Besides revealing the geometry of the factor analysis model, the model invariants also furnish useful statistics for testing goodness-of-fit.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Allman E.S., Rhodes J.A.: Phylogenetic ideals and varieties for the general Markov model. preprint, arXiv:math.AG/0410604
Anderson T.W., Rubin H.: Statistical inference in factor analysis. In: Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, vol. V, pp. 111–150. University of California Press, Berkeley and Los Angeles (1956)
Bekker P.A., de Leeuw J. (1987) The rank of reduced dispersion matrices. Psychometrika 52, 125–135
Catanese F., Hosten S., Khetan A., Sturmfels B. (2006) The maximum likelihood degree. Am. J. Math. 128, 671–697
Conca A. (1994) Gröbner bases of ideals of minors of a symmetric matrix. J. Algebra 166, 406–421
Cox D., Little J., O’Shea D. (1997) Ideals, Varieties, and Algorithms, 2nd edn. Springer, Berlin Heidelberg New York
Cox D.R., Wermuth N. (2002) On some models for multivariate binary variables parallel in complexity with the multivariate Gaussian distribution. Biometrika 89, 462–469
Ellis S. (2004) Instability of factor analysis. Proc. Am. Math. Soc. 132: 1805–1822
Geiger D., Heckerman D., King H., Meek C. (2001) Stratified exponential families: graphical models and model selection. Ann. Stat. 29, 505–529
Gel’fand I.M., Kapranov M.M., Zelevinsky A.V. (1994) Discriminants, Resultants and Multidimensional Determinants. Birkhäuser, Boston
Glymour C., Scheines R., Spirtes P., Kelly K. (1987) Discovering Causal Structure. Academic, London
Graczyk P., Letac G., Massam H. (2005) The hyperoctahedral group, symmetric group representations and the moments of the real Wishart distribution. J. Theor. Probab. 18, 1–42
Grayson D., Stillman M.: Macaulay 2, a software system for research in algebraic geometry. Available at http://www.math.uiuc.edu/Macaulay2/
Greuel G.-M., Pfister G., Schönemann H.: Singular 2.0. A computer algebra system for polynomial computations. Centre for Computer Algebra, University of Kaiserslautern (2001) http://www.singular.uni-kl.de
Grzebyk M., Wild P., Chouanière D. (2004) On identification of multi-factor models with correlated residuals. Biometrika 91, 141–151
Harman H. (1976) Modern Factor Analysis, 3rd edn. University of Chicago Press, Chicago
Hipp J.R., Bollen K.A. (2003) Model fit in structural equation models with censored, ordinal, and dichotomous variables: testing vanishing tetrads. Sociol. Methodol. 33, 267–305
Kelley T.L. (1935) Essential Traits of Mental Life, Harvard Studies in Education, vol 26. Harvard University Press, Cambridge
Lauritzen S.L. (1996) Graphical Models. Clarendon Press, Oxford
de Loera J.A., Sturmfels B., Thomas R.R. (1995) Gröbner bases and triangulations of the second hypersimplex. Combinatorica 15, 409–424
Lu I-L., Richards D.St.P. (2001) MacMahon’s master theorem, representation theory, and moments of Wishart distributions. Adv. Appl. Math. 27, 531–547
Mardia K.V., Kent J.T., Bibby J.M. (1979) Multivariate Analysis. Academic, London
Matus F. (2005) Conditional independences in Gaussian vectors and rings of polynomials. In: Kern-Isberner G., Rödder W., Kulmann F. (eds) Conditionals, Information, and Inference (WCII 2002 Hagen), Lecture Notes in Computer Science, vol. 3301. Springer, Berlin Heidelberg New York, pp. 152–161
Pachter L., Sturmfels B. (eds) (2005) Algebraic Statistics for Computational Biology. Cambridge University Press, Cambridge
Pistone G., Riccomagno E., Wynn H.P. (2001) Algebraic Statistics. Computational Commutative Algebra in Statistics. Chapman & Hall/CRC, Boca Raton
Roverato A., Whittaker J. (1998) The Isserlis matrix and its application to non-decomposable graphical Gaussian models. Biometrika 85, 711–725
Rubin D.B., Thayer D.T. (1982) EM algorithms for ML factor analysis. Psychometrika 47, 69–76
Santos F., Sturmfels B. (2003) Higher Lawrence configurations. J. Comb. Theory Ser. A 103, 151–164
Shorack G.R. (2000) Probability for Statisticians. Springer, Berlin Heidelberg New York
Spearman C. (1904) General intelligence, objectively determined and measured. Am. J. Psychol. 15, 201–293
Spearman C. (1927) The Abilities of Man. Macmillan, New York
Spirtes P., Glymour C., Scheines R. (2000) Causation, Prediction, and Search, 2nd edn. MIT Press, Cambridge
Sturmfels B. Introduction to resultants. In: Cox D., Sturmfels B. (eds) Applications of Computational Algebraic Geometry. Proceedings of Symposium in Applied Mathematics, vol 53, American Mathematical Society, pp. 25–39 (1997)
Sturmfels B.: Solving Systems of Polynomial Equations. American Mathematical Society, CBMS Lecture Series, No. 97, Providence, Rhode Island (2002)
Sturmfels B., Sullivant S.: Combinatorial secant varieties. arXiv:math.AC/0506223. Q J Pure Appl Math. 2, 285–309 (2006) (Special issue: In Honor of Robert MacPherson)
Watanabe S. (2001) Algebraic analysis for non-identifiable learning machines. Neural Comput. 13, 899–933
Wishart J. (1928a) The generalised product moment distribution in samples from a normal multivariate population. Biometrika 20A: 32–52
Wishart J. (1928b) Sampling errors in the theory of two factors. Br. J. Psychol. 19, 180–187
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Drton, M., Sturmfels, B. & Sullivant, S. Algebraic factor analysis: tetrads, pentads and beyond. Probab. Theory Relat. Fields 138, 463–493 (2007). https://doi.org/10.1007/s00440-006-0033-2
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00440-006-0033-2