The authors consider the two- and three-dimensional instability of finite-amplitude standing waves in a rectangular and a circular basin. For the stability analysis the authors apply the transition matrix method for unsteady flows. The method requires not only the precise determination of the nonlinear base flow, but also the calculation of the transition matrix to high accuracy. To accomplish this, the authors develop a new high-order spectral element method which is an extension of the high-order spectral method for nonlinear wave interactions. Moreover, they show that the instabilities are direct consequences of quartet internal resonances, and offer a possible explanation for the transition of standing waves in circular tanks from axial symmetry to non-axisymmetry observed in experiments.