The concept of angles between lines (in the plane or in space), between two planes (dihedral angle) or between a line and a plane can be generalized to arbitrary dimensions. This generalization was first discussed by Camille Jordan.[1] For any pair of flats in a Euclidean space of arbitrary dimension one can define a set of mutual angles which are invariant under isometric transformation of the Euclidean space. If the flats do not intersect, their shortest distance is one more invariant.[1] These angles are called canonical[2] or principal.[3] The concept of angles can be generalized to pairs of flats in a finite-dimensional inner product space over the complex numbers.

Jordan's definition

edit

Let   and   be flats of dimensions   and   in the  -dimensional Euclidean space  . By definition, a translation of   or   does not alter their mutual angles. If   and   do not intersect, they will do so upon any translation of   which maps some point in   to some point in  . It can therefore be assumed without loss of generality that   and   intersect.

Jordan shows that Cartesian coordinates             in   can then be defined such that   and   are described, respectively, by the sets of equations

 
 
 

and

 
 
 

with  . Jordan calls these coordinates canonical. By definition, the angles   are the angles between   and  .

The non-negative integers   are constrained by

 
 
 

For these equations to determine the five non-negative integers completely, besides the dimensions   and   and the number   of angles  , the non-negative integer   must be given. This is the number of coordinates  , whose corresponding axes are those lying entirely within both   and  . The integer   is thus the dimension of  . The set of angles   may be supplemented with   angles   to indicate that   has that dimension.

Jordan's proof applies essentially unaltered when   is replaced with the  -dimensional inner product space   over the complex numbers. (For angles between subspaces, the generalization to   is discussed by Galántai and Hegedũs in terms of the below variational characterization.[4])[1]

Angles between subspaces

edit

Now let   and   be subspaces of the  -dimensional inner product space over the real or complex numbers. Geometrically,   and   are flats, so Jordan's definition of mutual angles applies. When for any canonical coordinate   the symbol   denotes the unit vector of the   axis, the vectors       form an orthonormal basis for   and the vectors       form an orthonormal basis for  , where

 

Being related to canonical coordinates, these basic vectors may be called canonical.

When   denote the canonical basic vectors for   and   the canonical basic vectors for   then the inner product   vanishes for any pair of   and   except the following ones.

 

With the above ordering of the basic vectors, the matrix of the inner products   is thus diagonal. In other words, if   and   are arbitrary orthonormal bases in   and   then the real, orthogonal or unitary transformations from the basis   to the basis   and from the basis   to the basis   realize a singular value decomposition of the matrix of inner products  . The diagonal matrix elements   are the singular values of the latter matrix. By the uniqueness of the singular value decomposition, the vectors   are then unique up to a real, orthogonal or unitary transformation among them, and the vectors   and   (and hence  ) are unique up to equal real, orthogonal or unitary transformations applied simultaneously to the sets of the vectors   associated with a common value of   and to the corresponding sets of vectors   (and hence to the corresponding sets of  ).

A singular value   can be interpreted as   corresponding to the angles   introduced above and associated with   and a singular value   can be interpreted as   corresponding to right angles between the orthogonal spaces   and  , where superscript   denotes the orthogonal complement.

Variational characterization

edit

The variational characterization of singular values and vectors implies as a special case a variational characterization of the angles between subspaces and their associated canonical vectors. This characterization includes the angles   and   introduced above and orders the angles by increasing value. It can be given the form of the below alternative definition. In this context, it is customary to talk of principal angles and vectors.[3]

Definition

edit

Let   be an inner product space. Given two subspaces   with  , there exists then a sequence of   angles   called the principal angles, the first one defined as

 

where   is the inner product and   the induced norm. The vectors   and   are the corresponding principal vectors.

The other principal angles and vectors are then defined recursively via

 

This means that the principal angles   form a set of minimized angles between the two subspaces, and the principal vectors in each subspace are orthogonal to each other.

Examples

edit

Geometric example

edit

Geometrically, subspaces are flats (points, lines, planes etc.) that include the origin, thus any two subspaces intersect at least in the origin. Two two-dimensional subspaces   and   generate a set of two angles. In a three-dimensional Euclidean space, the subspaces   and   are either identical, or their intersection forms a line. In the former case, both  . In the latter case, only  , where vectors   and   are on the line of the intersection   and have the same direction. The angle   will be the angle between the subspaces   and   in the orthogonal complement to  . Imagining the angle between two planes in 3D, one intuitively thinks of the largest angle,  .

Algebraic example

edit

In 4-dimensional real coordinate space R4, let the two-dimensional subspace   be spanned by   and  , and let the two-dimensional subspace   be spanned by   and   with some real   and   such that  . Then   and   are, in fact, the pair of principal vectors corresponding to the angle   with  , and   and   are the principal vectors corresponding to the angle   with  

To construct a pair of subspaces with any given set of   angles   in a   (or larger) dimensional Euclidean space, take a subspace   with an orthonormal basis   and complete it to an orthonormal basis   of the Euclidean space, where  . Then, an orthonormal basis of the other subspace   is, e.g.,

 

Basic properties

edit
  • If the largest angle is zero, one subspace is a subset of the other.
  • If the largest angle is  , there is at least one vector in one subspace perpendicular to the other subspace.
  • If the smallest angle is zero, the subspaces intersect at least in a line.
  • If the smallest angle is  , the subspaces are orthogonal.
  • The number of angles equal to zero is the dimension of the space where the two subspaces intersect.

Advanced properties

edit
  • Non-trivial (different from   and   [5]) angles between two subspaces are the same as the non-trivial angles between their orthogonal complements.[6][7]
  • Non-trivial angles between the subspaces   and   and the corresponding non-trivial angles between the subspaces   and   sum up to  .[6][7]
  • The angles between subspaces satisfy the triangle inequality in terms of majorization and thus can be used to define a distance on the set of all subspaces turning the set into a metric space.[8]
  • The sine of the angles between subspaces satisfy the triangle inequality in terms of majorization and thus can be used to define a distance on the set of all subspaces turning the set into a metric space.[6] For example, the sine of the largest angle is known as a gap between subspaces.[9]

Extensions

edit

The notion of the angles and some of the variational properties can be naturally extended to arbitrary inner products[10] and subspaces with infinite dimensions.[7]

Computation

edit

Historically, the principal angles and vectors first appear in the context of canonical correlation and were originally computed using SVD of corresponding covariance matrices. However, as first noticed in,[3] the canonical correlation is related to the cosine of the principal angles, which is ill-conditioned for small angles, leading to very inaccurate computation of highly correlated principal vectors in finite precision computer arithmetic. The sine-based algorithm[3] fixes this issue, but creates a new problem of very inaccurate computation of highly uncorrelated principal vectors, since the sine function is ill-conditioned for angles close to π/2. To produce accurate principal vectors in computer arithmetic for the full range of the principal angles, the combined technique[10] first compute all principal angles and vectors using the classical cosine-based approach, and then recomputes the principal angles smaller than π/4 and the corresponding principal vectors using the sine-based approach.[3] The combined technique[10] is implemented in open-source libraries Octave[11] and SciPy[12] and contributed [13] and [14] to MATLAB.

See also

edit

References

edit
  1. ^ a b c Jordan, C. (1875). "Essai sur la géométrie à   dimensions". Bull. Soc. Math. France. 3: 103.
  2. ^ Afriat, S. N. (1957). "Orthogonal and oblique projectors and the characterization of pairs of vector spaces". Math. Proc. Cambridge Philos. Soc. 53 (4): 800. doi:10.1017/S0305004100032916. S2CID 122049149.
  3. ^ a b c d e Björck, Å.; Golub, G. H. (1973). "Numerical Methods for Computing Angles Between Linear Subspaces". Math. Comp. 27 (123): 579. doi:10.2307/2005662. JSTOR 2005662.
  4. ^ Galántai, A.; Hegedũs, Cs. J. (2006). "Jordan's principal angles in complex vector spaces". Numer. Linear Algebra Appl. 13 (7): 589–598. CiteSeerX 10.1.1.329.7525. doi:10.1002/nla.491. S2CID 13107400.
  5. ^ Halmos, P.R. (1969), "Two subspaces", Trans. Amer. Math. Soc., 144: 381–389, doi:10.1090/S0002-9947-1969-0251519-5
  6. ^ a b c Knyazev, A.V.; Argentati, M.E. (2006), "Majorization for Changes in Angles Between Subspaces, Ritz Values, and Graph Laplacian Spectra", SIAM J. Matrix Anal. Appl., 29 (1): 15–32, CiteSeerX 10.1.1.331.9770, doi:10.1137/060649070, S2CID 16987402
  7. ^ a b c Knyazev, A.V.; Jujunashvili, A.; Argentati, M.E. (2010), "Angles between infinite dimensional subspaces with applications to the Rayleigh–Ritz and alternating projectors methods", Journal of Functional Analysis, 259 (6): 1323–1345, arXiv:0705.1023, doi:10.1016/j.jfa.2010.05.018, S2CID 5570062
  8. ^ Qiu, L.; Zhang, Y.; Li, C.-K. (2005), "Unitarily invariant metrics on the Grassmann space" (PDF), SIAM Journal on Matrix Analysis and Applications, 27 (2): 507–531, doi:10.1137/040607605
  9. ^ Kato, D.T. (1996), Perturbation Theory for Linear Operators, Springer, New York
  10. ^ a b c Knyazev, A.V.; Argentati, M.E. (2002), "Principal Angles between Subspaces in an A-Based Scalar Product: Algorithms and Perturbation Estimates", SIAM Journal on Scientific Computing, 23 (6): 2009–2041, Bibcode:2002SJSC...23.2008K, CiteSeerX 10.1.1.73.2914, doi:10.1137/S1064827500377332
  11. ^ Octave function subspace
  12. ^ SciPy linear-algebra function subspace_angles
  13. ^ MATLAB FileExchange function subspace
  14. ^ MATLAB FileExchange function subspacea