The central problem of the paper is a characterization of extreme and exposed points of unit balls of the Paley-Wiener space \(PW^1_S\), the subspace of \(L^1(\mathbb{R})\) consisting of functions whose spectra (the supports of their Fourier transforms) are contained in \(S\subset \mathbb{R}\).
When \(S\) is a symmetric interval \([-\sigma,\sigma]\) the problem has been resolved by K. M. Dyakonov [Math. Res. Lett. 7, No. 4, 393–404 (2000; Zbl 0973.30006)]. The authors of this paper study the problem when \(S\) is an interval with a gap, \(S=[-\sigma,\sigma]\setminus (-\rho,\rho)\).
When the gap interval \((-\rho,\rho)\) is large enough, the authors give a complete characterization of both extreme and exposed points of the unit ball in \(PW^1_S\), using a (unique) decomposition of \(f\in PW^1_{S}\), \(f=f_-+f_+\), where the spectra of \(f_-\) and \(f_+\) are subsets of \([-\sigma,-\rho]\) and \([\rho,\sigma]\) respectively. The main Theorem 1.2 is paraphrased below.
Theorem: Assume that \(2\rho>\sigma\)

●

the extreme points of the unit ball in \(PW^1_S\) are precisely the functions \(f\in PW^1_{S}\) such that \(\|f\|_{1} = 1\), at least one of the points \(\sigma\), \(-\sigma\), \(\rho,-\rho\) belongs to the spectrum of \(f\), and neither \(f_-\) nor \(f_+\) being extended to \(\mathbb{C}\) have (non-real) zeros symmetric with the real axis.

●

the exposed points of the unit ball in \(PW^1_S\) are precisely the functions \(f\in PW^1_{S}\) such that neither \(f_-\) nor \(f_+\) have real zeros of order \(2\) or higher, and \(f w\notin L^1(\mathbb{R})\) for every entire real (on \(\mathbb{R}\)) non-constant function \(w\) of zero type.

In Theorem 1.3 the authors show that the set of extreme points of the unit ball is dense on the unit sphere in \(PW^1_S\) when the gap interval is large. It is also shown that the assumption \(2\rho>\sigma\) is crucial for both parts of the main theorem. Theorems 1.4 and 1.5 assert the existence of various counter-examples when \(\rho\) is small.