Let \(H\) be a finite-dimensional non-zero complex Hilbert space. A family \({\mathcal N}\) of subspaces of \(H\) is called a nest if it is totally-ordered by the set inclusion. Let \(\{f_1, \ldots, f_n\}\) be a basis of \(H\) and \(N_i=span \{f_1, \ldots, f_i\}\) then \({\mathcal N}=\{(0)\} \cup \{N_i \colon 1\leq i \leq n \}\) is called a maximal nest. In this case it is said that the nest \({\mathcal N}\) arises from the basis \(\{f_1, \ldots, f_n\}\). For any nest \({\mathcal N}\) the nest algebra \(\text{Alg} {\mathcal N}\) consists of all operators which leave each element of \({\mathcal N}\) invariant.
Let \({\mathcal M}\) and \({\mathcal N}\) be maximal nests. It is shown that the dimension of the intersection of the corresponding nest algebras is at least \(\dim H\). Moreover, an example of three maximal nests whose algebras intersect minimally, that is the intersection is the set of scalar operators, is found. The authors show that the dimension of the intersections of two nest algebras (corresponding to maximal nests) can be of any integer value between \(n\) and \(n(n+1)/2\), where \(n=\dim H\). For a given basis \(\{f_1, \ldots, f_n\}\) of \(H\), and for a given family of permutations \(\{ \pi_i: 1\leq i \leq k\}\) of \(\{1,2,\ldots, n\}\) with \(\pi_1=\text{ identity}\), one can obtain a family \(\{ {\mathcal N}_i : 1\leq i \leq k \}\) of maximal nests by taking \({\mathcal N}_i\) to be the nest arising from the basis \(\{f_{\pi _i(1)}, \ldots, f_{\pi _i(n)}\}\). Such families \(\{ {\mathcal N}_i : 1\leq i \leq k \}\) are characterized in terms of dimensions of certain intersections. As a corollary it is proved that for any two maximal nests, there exists a basis \(\{f_1, \ldots, f_n\}\) of \(H\) and a permutation \(\pi\in S_n\) such that these nests arise from the bases \(\{f_1, \ldots, f_n\}\) and \(\{f_{\pi (1)}, \ldots, f_{\pi (n)}\}\), correspondingly. Those pairs of maximal nests for which the intersections of the corresponding nest algebras have minimum dimension are characterized. Namely, it is proved that there exists a basis \(\{f_1, \ldots, f_n\}\) of \(H\) such that the nests under consideration arise from the bases \(\{f_1, \ldots, f_n\}\) and \(\{f_n,f_{n-1},f_{n-2},\dots, f_1\}\), correspondingly.
The developed theory is applied the incidence algebras. Incidence algebras on finite sets are characterized as intersections of certain nest algebras. Additionally it is shown that any incidence algebra on a finite set is a subalgebra of certain nest algebras which contains all elements whose matrix relative to the basis, which determines the maximality of the corresponding nest, is diagonal. Here, \(\dim H\) is equal to the number of elements in the set determining the algebra.