Let \(G/K\) be a compact homogeneous space, where \(G\) is a compact is group and \(K\) is a closed subgroup of \(G\), and let \(G^{\mathbb C}/K^{\mathbb C}\) be its Lie group complexification. Denote by \(\widehat G\) the image of \(G\) in \(\mathrm{Aut}(G^{\mathbb C}/K^{\mathbb C})\), i.e., the quotient group of \(G\) by its ineffectiveness on \(G^{\mathbb C}/K^{\mathbb C}\). In this paper the authors investigate the group of holomorphic automorphisms of a \(G\)-invariant domain \(D\) in \(G^{\mathbb C}/K^{\mathbb C}\). The first result of the paper relates it to the isometry group of the space \(G/K\).
Theorem. Let \(D\subset G^{\mathbb C}/K^{\mathbb C}\) be \(G\)-invariant domain and let \(K\) be connected. If \(W\) is a connected compact subgroup of \(\mathrm{Aut}(D)\) containing \(\widehat G\), then \(W\) can be realized as a subgroup of the isometry group \(\mathrm{Iso}(G/K,g)\) of the space \( G/K\) endowed with some \(G\)-invariant Riemannian metric \(g\). Furthermore, if \(D\) is relatively compact in \(G^{\mathbb C}/K^{\mathbb C}\), then the identity component \(\mathrm{Aut}(D)^0\) of \(\mathrm{Aut}(D)\) can be realized as a closed subgroup of \(\mathrm{Iso}(G/K,g)\).
Then the following rigidity results are obtained:
Theorem. Let \(G/K\) be a strongly isotropy irreducible homogeneous space, where \(G\) is a compact connected Lie group with universal covering different from \(G_2\) and \(\mathrm{Spin}(7)\), and \(K\) is a closed subgroup of \(G\). Let \(D\) be a hyperbolic \(G\)-invariant domain in \(G^{\mathbb C}/K^{\mathbb C}\). Then \(\mathrm{Aut}(D)^0=\widehat G\).
Theorem. Let \(G/K\) be an isotropy irreducible homogeneous space, where \(G\) is a compact connected Lie group and \(K\) is a closed subgroup of \(G\) with \(\dim K>0\). Let \(D\) be a \(G\)-invariant relatively compact domain or a \(G\)-invariant hyperbolic Stein domain in \(G^{\mathbb C}/K^{\mathbb C}\). Then \(v(D)^0=\widehat G\).
The above theorems improve results previously obtained in [X.-Y. Zhou, Ann. Inst. Fourier 47, No. 4, 1101–1015 (1997; Zbl 0881.32015)] and [G. Fels and L. Geatti, J. Reine Angew. Math. 454, 97–118 (1994; Zbl 0803.32019)].