Let \(G\) be a locally compact group. For a unitary representation \(\pi\) of \(G\), let \(B_\pi(G)\) denote the \(w^*\)-closed linear subspace of the Fourier-Stieltjes algebra \(B(G)\) generated by all coefficients of \(\pi\), and \(B^0_\pi(G)\) the closure of \(B_\pi(G)\cap A_c(G)\), where \(A_c(G)\) consists of all functions in the Fourier algebra \(A(G)\) with compact support. The authors present the following descriptions of \(B_\pi^0(G)\) and its orthogonal complement \(B^s_\pi(G)\) in \(B_\pi(G)\):
(i) \(u\in B^0_\pi(G)\) if and only if for every \(\varepsilon >0\) there exists a compact subset \(K\) of \(G\) such that \(|\langle u,f\rangle |< \varepsilon\) for all \(f\in L^1(G\setminus K)\) with \(|\pi(f)\|\leq 1\); (ii) \(u\in B^s_\pi(G)\) if and only if for every \(\varepsilon >0\) and each compact subset \(K\) of \(G\) there exists \(f\in L^1(G \setminus K)\) such that \(|\langle u,f\rangle |>\|u\|-\varepsilon\) and \(\|\pi(f) \|\leq 1\).
This generalizes a recent result of the reviewer. Let \({\mathcal B}^0_\pi (G)=A_c(G) +B^0_\pi(G)\). The major portion of the paper is devoted to the study of how \(B^0_\pi(G)\) and \({\mathcal B}^0_\pi(G)\) depend on \(\pi\) and \(G\). It is proved that, for some classes of locally compact groups \(G\), there is a dichotomy in the sense that for any \(\pi\) either \(B^0_\pi(G)= \{0\}\) or \(B^0_\pi(G) =A(G)\). The authors also characterize functions in \({\mathcal B}^0_\pi(G)\) and study the question of whether \({\mathcal B}^0_\pi (G)=A(G)\) implies that \(\pi\) weakly contains the regular representation \(\rho\). Several results are obtained in this paper which strongly support the conjecture that this question has an affirmative answer in general.