The authors consider the problem
\[\begin{cases}-F(D^2 u)=\vert u\vert^{p-1}u \quad&\text{in } B,\\ u=0&\text{on } \partial B,\\ u(0)>0&\text{in }B,\end{cases}\]
where \(B\) is the unit ball of \({\mathbb R}^N\), \(p>1\), \(0<\lambda\leq\Lambda\) and \(F\) is either one of the Pucci’s extremal operators \(M^\pm_{\lambda,\Lambda}\), defined by
\[M^-_{\lambda,\Lambda}(X)=\inf_{\lambda I\leq A\leq \Lambda I} tr(AX)=\lambda\sum_{\mu_i>0}\mu_i+\Lambda\sum_{\mu_i<0}\mu_i,\] \[M^+_{\lambda,\Lambda}(X)=\sup_{\lambda I\leq A\leq \Lambda I} tr(AX)=\Lambda\sum_{\mu_i>0}\mu_i+\lambda\sum_{\mu_i<0}\mu_i,\]
\(\mu_1\),...,\(\mu_N\) being the eigenvalues of any squared symmetric matrix \(X\).
Previously, in the paper [Ann. Inst. Henri Poincaré, Anal. Non Linéaire 20, No. 5, 843–865 (2003; Zbl 1274.35115)] P. L. Felmer and A. Quaas proved that there exist two critical exponents \(p^*_-<p^*_+\) (not explicitly known) such that positive radial classical solutions to the previous problem exist if and only if \(p<p^*_-\) for \(F= M^-_{\lambda,\Lambda}\) or \(p<p^*_+\) for \(F= M^+_{\lambda,\Lambda}\).
The main result of the present paper shows that such a bound on the exponent \(p\) is not optimal if radial sign-changing solutions are considered. Indeed they obtain the following theorem:
i) If \(F= M^-_{\lambda,\Lambda}\), then radial sign-changing solutions of the previous problem with any number of nodal domains exist if and only if \(p<p^*_-\).
ii) If \(F= M^+_{\lambda,\Lambda}\), then there exists a new critical exponent \(p_+^{**}\) satisfying \(p^{**}_+<p<p^*_+\) such that no radial sign-changing solutions of the previous problem exist for \(p\geq p^{**}_+\), while radial sign-changing solutions with any number of nodal domains exist at least for a sequence of exponents \(p_n\uparrow p^{**}_+\).
After proving this result, the authors analyze the asymptotic behavior of the radial nodal solutions as the exponents approach the critical values, showing that new concentration phenomena occur. Finally they define a suitable weighted energy for these solutions and compute its limit value.