×

Nilpotent orbits and some small unitary representations of indefinite orthogonal groups. (English) Zbl 1082.22009

Summary: For \(2 \leq m \leq 1/2\), let \(G\) be a simply connected Lie group with Lie algebra \(g_0 = so(2m,2l-2m)\), let \(g = k \oplus p\) be the complexification of the usual Cartan decomposition, let \(K\) be the analytic subgroup with Lie algebra \(k \cap g_0\), and let \(U (g)\) be the universal enveloping algebra of \(g\). This work examines the unitarity and \(K\) spectrum of representations in the “analytic continuation” of discrete series of \(G\), relating these properties to orbits in the nilpotent radical of a certain parabolic subalgebra of \(g\).
The roots with respect to the usual compact Cartan subalgebra are all \(\pm e_i \pm e_j\) with \(1 \leq i < j \leq 1\). In the usual positive system of roots, the simple root \(e_m-e_{m+1}\) is noncompact and the other simple roots are compact. Let \(q = 1 \oplus u\) be the parabolic subalgebra of \(g\) for which \(e_m-e_{m+1}\) contributes to \(u\) and the other simple roots contribute to 1, let \(L\) be the analytic subgroup of \(G\) with Lie algebra \(1 \cap g_0\), let \(L^C = \text{Int}_g(l)\), let \(2\delta(u)\) be the sum of the roots contributing to \(u\), and let \(\overline{q}=l \oplus \overline{u}\) be the parabolic subalgebra opposite to \(q\).
The members of \(u \cap p\) are nilpotent members of \(g\). The group \(L^C\) acts on \(u \cap p\) with finitely many orbits, and the topological closure of each orbit is an irreducible algebraic variety. If \(Y\) is one of these varieties, let \(R(Y)\) be the dual coordinate ring of \(Y\); this is a quotient of the algebra of symmetric tensors on \(u \cap p\) that carries a fully reducible representation of \(L^C\).
For an integer \(s\), let \(\lambda_s = \sum_{k=1}^m (-1+s/2)e_k\). Then \(\lambda_s\) defines a one-dimensional \((1,L)\) module \(C_{\lambda_s}\). Extend this to a \((\overline{q},L)\) module by having \(\overline{u}\) act by 0, and define \(N(\lambda_s+2\delta(u)) = U(g) \otimes_{\overline{q}} C_{\lambda_s+2\delta(u)}\). Let \(N'(\lambda_s+2\delta(u))\) be the unique irreducible quotient of \(N(\lambda_s+2\delta(u))\). The representations under study are \(\pi_s = \Pi_S (N(\lambda_s+2\delta(u)))\) and \(\pi'_s=\Pi_S(N'(\lambda_s+2\delta(u)))\), where \(S = \dim(u \cap k)\) and \(\Pi_S\) is the \(S\)th derived Bernstein functor.
For \(s > 2l-2\), it is known that \(\pi_s = \pi'_s\) and that \(\pi'_s\) is in the discrete series. Enright, Parthasarathy, Wallach, and Wolf showed for \(m \leq s \leq 2l-2\) that \(\pi_s = \pi'_s\) and that \(\pi'_s\) is still unitary. The present paper shows that \(\pi'_s\) is unitary for \(0 \leq s \leq m-1\) even though \(\pi_s \neq \pi'_s\), and it relates the \(K\) spectrum of the representations \(\pi'_s\) to the representation of \(L^C\) on a suitable \(R(Y)\) with \(Y\) depending on \(s\). Use of a branching formula of D. E. Littlewood allows one to obtain an explicit multiplicity formula for each \(K\) type in \(\pi'_s\). The chieftools involved are an idea of \(B \). Gross and Wallach, a geometric interpretation of Littlewood’s theorem, and some estimates of norms.
It is shown further that the natural invariant Hermitian form on \(\pi'_s\) does not make \(\pi'_s\) unitary for \(s < 0\) and that the \(K\) spectrum of \(\pi'_s\) in these cases is not related in the above way to the representation of \(L^C\) on any \(R(Y)\).
A final section of the paper treats in similar fashion the simply connected Lie group with Lie algebra \(g_0=\text{so}(2m,2l-2n+1)\), \(2 \leq m \leq 1/2\).

MSC:

22E45 Representations of Lie and linear algebraic groups over real fields: analytic methods
20G20 Linear algebraic groups over the reals, the complexes, the quaternions
14L35 Classical groups (algebro-geometric aspects)
Full Text: DOI

References:

[1] Binegar, B.; Zierau, R., Unitarization of a singular representation of \(SO (p,q)\), Comm. Math. Phys., 138, 245-258 (1991) · Zbl 0748.22009
[2] Borel, A.; de Siebenthal, J., Les sous-groupes fermés de rang maximum des groupes de Lie clos, Comment. Math. Helv., 23, 200-221 (1949) · Zbl 0034.30701
[3] Brylinski, R.; Kostant, B., Minimal representations, geometric quantization, and unitarity, Proc. Nat. Acad. Sci. USA, 91, 6026-6029 (1994) · Zbl 0803.58023
[4] deConcini, C.; Procesi, C., A characteristic free approach to classical invariant theory, Adv. Math., 21, 330-354 (1976) · Zbl 0347.20025
[5] Enright, T.; Howe, R.; Wallach, N., A classification of unitary highest weight modules, (Trombi, P. C., Representation Theory of Reductive Groups (1983), Birkhäuser: Birkhäuser Boston), 97-143 · Zbl 0535.22012
[6] Enright, T. J.; Parthasarathy, R.; Wallach, N. R.; Wolf, J. A., Unitary derived functor modules with small spectrum, Acta Math., 154, 105-136 (1985) · Zbl 0568.22007
[7] R. Goodman, N.R. Wallach, Representations and Invariants of the Classical Groups. Encyclopedia of Mathematics and Its Applications, Vol. 68, Cambridge University Press, Cambridge, 1998.; R. Goodman, N.R. Wallach, Representations and Invariants of the Classical Groups. Encyclopedia of Mathematics and Its Applications, Vol. 68, Cambridge University Press, Cambridge, 1998. · Zbl 0901.22001
[8] S. Greenleaf, Decompositions of group actions on symmetric tensors, Thesis, State University of New York, Stony Brook, 2000.; S. Greenleaf, Decompositions of group actions on symmetric tensors, Thesis, State University of New York, Stony Brook, 2000.
[9] Gross, B. H.; Wallach, N. R., A distinguished family of unitary representations for the exceptional groups of real rank=4, (Brylinski, J.-L.; Brylinski, R.; Guillemin, V.; Kac, V., Lie Theory and Geometry: in Honor of Bertram Kostant (1994), Birkhäuser: Birkhäuser Boston), 289-304 · Zbl 0839.22006
[10] Gross, B. H.; Wallach, N. R., On quaternionic discrete series representations, and their continuations, J. Reine Angew. Math., 481, 73-123 (1996) · Zbl 0857.22012
[11] Harish-Chandra, Discrete series for semisimple Lie groups II, Acta Math. 116 (1966) 1-111.; Harish-Chandra, Discrete series for semisimple Lie groups II, Acta Math. 116 (1966) 1-111. · Zbl 0199.20102
[12] R. Howe, Perspectives on invariant theory, The Schur Lectures (1992), Israel Mathematics Conference Proceedings, Vol. 8, American Mathematical Society, Providence, RI, 1995, pp. 1-182.; R. Howe, Perspectives on invariant theory, The Schur Lectures (1992), Israel Mathematics Conference Proceedings, Vol. 8, American Mathematical Society, Providence, RI, 1995, pp. 1-182. · Zbl 0844.20027
[13] Jakobsen, H. P., Hermitian symmetric spaces and their unitary highest weight modules, J. Funct. Anal., 52, 385-412 (1983) · Zbl 0517.22014
[14] Kazhdan, D.; Savin, G., The smallest representations of simply laced groups, (Gelbart, S.; Howe, R.; Sarnak, P., Festschrift in Honor of I. I. Piatetski-Shapiro on the Occasion on His Sixtieth Birthday, Part I. Festschrift in Honor of I. I. Piatetski-Shapiro on the Occasion on His Sixtieth Birthday, Part I, Israel Mathematics Conference Proceedings, Vol. 2 (1990), Weizmann Science Press of Israel: Weizmann Science Press of Israel Jerusalem), 209-223 · Zbl 0737.22008
[15] Knapp, A. W., Exceptional unitary representations of semisimple Lie groups, Represent. Theory, 1, 1-24 (1997) · Zbl 0887.22019
[16] A.W. Knapp, Lie Groups Beyond an Introduction, Birkhäuser, Boston, 1996; 2nd edition, 2002.; A.W. Knapp, Lie Groups Beyond an Introduction, Birkhäuser, Boston, 1996; 2nd edition, 2002. · Zbl 0862.22006
[17] A.W. Knapp, Analytic continuation of nonholomorphic discrete series for classical groups, in: Noncommutative Harmonic Analysis in Honor of Jacques Carmona, Birkhäuser, Boston, 2003, pp. 253-289.; A.W. Knapp, Analytic continuation of nonholomorphic discrete series for classical groups, in: Noncommutative Harmonic Analysis in Honor of Jacques Carmona, Birkhäuser, Boston, 2003, pp. 253-289. · Zbl 1062.22033
[18] Knapp, A. W., Geometric interpretations of two branching theorems of D. E. Littlewood, J. Algebra, 270, 728-754 (2003) · Zbl 1038.22005
[19] Knapp, A. W.; Vogan, D. A., Cohomological Induction and Unitary Representations (1995), Princeton University Press: Princeton University Press Princeton, NJ · Zbl 0863.22011
[20] Kobayashi, T., Singular unitary representations and discrete series for indefinite Stiefel manifolds \(U(p,q;F)/U(p−m,q;F)\), Mem. Amer. Math. Soc., 95, No. 462, vi+106 pp (1992) · Zbl 0752.22007
[21] B. Kostant, The principle of triality and a distinguished unitary representation of SO(4,4), Differential Geometrical Methods in Theoretical Physics (Como, 1987), NATO Advanced Science Institutes Series C: Mathematical and Physical Sciences, Vol. 250, Kluwer, Dordrecht, 1988, pp. 65-108.; B. Kostant, The principle of triality and a distinguished unitary representation of SO(4,4), Differential Geometrical Methods in Theoretical Physics (Como, 1987), NATO Advanced Science Institutes Series C: Mathematical and Physical Sciences, Vol. 250, Kluwer, Dordrecht, 1988, pp. 65-108. · Zbl 0663.22009
[22] B. Kostant, The vanishing of scalar curvature and the minimal representation of SO(4,4), Operator Algebras, Unitary Representations, Enveloping Algebras, and Invariant Theory (Paris, 1989), Birkhäuser, Boston, 1990, pp. 85-124.; B. Kostant, The vanishing of scalar curvature and the minimal representation of SO(4,4), Operator Algebras, Unitary Representations, Enveloping Algebras, and Invariant Theory (Paris, 1989), Birkhäuser, Boston, 1990, pp. 85-124. · Zbl 0739.22012
[23] Li, Jian-Shu, Singular unitary representations of classical groups, Invent. Math., 97, 237-255 (1989) · Zbl 0694.22011
[24] Li, Jian-Shu, On the classification of irreducible low rank unitary representations of classical groups, Compositio Math., 71, 29-48 (1989) · Zbl 0694.22012
[25] D.E. Littlewood, The Theory of Group Characters and Matrix Representations of Groups, Oxford University Press, New York, 1940; 2nd Edition, 1950.; D.E. Littlewood, The Theory of Group Characters and Matrix Representations of Groups, Oxford University Press, New York, 1940; 2nd Edition, 1950. · Zbl 0025.00901
[26] I.G. Macdonald, Symmetric Functions and Hall Polynomials, Clarendon Press, Oxford 1979; 2nd Edition, 1995.; I.G. Macdonald, Symmetric Functions and Hall Polynomials, Clarendon Press, Oxford 1979; 2nd Edition, 1995. · Zbl 0487.20007
[27] Maliakas, M., Representation theoretic realizations of two classical character formulas of D. E. Littlewood, Comm. Algebra, 19, 271-296 (1991) · Zbl 0719.20021
[28] Maliakas, M., The universal form of the branching rule for the symplectic groups, J. Algebra, 168, 221-248 (1994) · Zbl 0829.20060
[29] Murnaghan, F. D., The Theory of Group Representations (1938), Johns Hopkins Press: Johns Hopkins Press Baltimore · Zbl 0022.11807
[30] Newell, M. J., Modification rules for the orthogonal and symplectic groups, Proc. Roy. Irish Acad. Sect. A, 54, 153-163 (1951) · Zbl 0044.25802
[31] K. Nishiyama, H. Ochiai, K. Taniguchi, H. Yamashita, Nilpotent orbits, associated cycles and Whittaker modules for highest weight representations, Astérisque 273 (2001) vi+163 pp.; K. Nishiyama, H. Ochiai, K. Taniguchi, H. Yamashita, Nilpotent orbits, associated cycles and Whittaker modules for highest weight representations, Astérisque 273 (2001) vi+163 pp. · Zbl 0968.22001
[32] Noël, A. G., Nilpotent orbits and theta-stable parabolic subalgebras, Represent. Theory, 2, 1-32 (1998) · Zbl 0891.17006
[33] H. Rubenthaler, Algèbres de Lie et Espaces Préhomogènes, Travaux en Cours, Vol. 44, Hermann, Paris, 1992.; H. Rubenthaler, Algèbres de Lie et Espaces Préhomogènes, Travaux en Cours, Vol. 44, Hermann, Paris, 1992. · Zbl 0840.17007
[34] Sato, M.; Kimura, T., A classification of irreducible prehomogeneous vector spaces and their relative invariants, Nagoya Math. J., 65, 1-155 (1977) · Zbl 0321.14030
[35] Schmid, W., Die Randwerte homomorpher Funktionen auf hermitesch symmetrischen Räumen, Invent. Math., 9, 61-80 (1969) · Zbl 0219.32013
[36] Strichartz, R. S., Harmonic analysis on hyperboloids, J. Funct. Anal., 12, 341-383 (1973) · Zbl 0253.43013
[37] E.B. Vinberg, On the classification of the nilpotent elements of graded Lie algebras, Dokl. Acad. Nauk SSSR 225 (1975) 745-748 (in Russian) (English translation, Soviet Math. Doklady 16 (1975) 1517-1520).; E.B. Vinberg, On the classification of the nilpotent elements of graded Lie algebras, Dokl. Acad. Nauk SSSR 225 (1975) 745-748 (in Russian) (English translation, Soviet Math. Doklady 16 (1975) 1517-1520). · Zbl 0374.17001
[38] D.A. Vogan, Singular unitary representations, Non Commutative Harmonic Analysis and Lie Groups, Lecture Notes in Mathematics, Vol. 880, Springer, Berlin, 1981, pp. 506-535.; D.A. Vogan, Singular unitary representations, Non Commutative Harmonic Analysis and Lie Groups, Lecture Notes in Mathematics, Vol. 880, Springer, Berlin, 1981, pp. 506-535. · Zbl 0464.22007
[39] Vogan, D. A., Unitarizability of certain series of representations, Ann. of Math., 120, 141-187 (1984) · Zbl 0561.22010
[40] Wallach, N., The analytic continuation of the discrete series I, Trans. Amer. Math. Soc., 251, 1-17 (1979) · Zbl 0419.22017
[41] Wallach, N., The analytic continuation of the discrete series II, Trans. Amer. Math. Soc., 251, 19-37 (1979) · Zbl 0419.22018
[42] Zhu, Chen-Bo; Huang, Jing-Song, On certain small representations of indefinite orthogonal groups, Represent. Theory, 1, 190-206 (1997) · Zbl 0887.22016
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.