Summary: We describe the continuous operator method for solution nonlinear operator equations and discuss its application for investigating direct and inverse scattering problems. The continuous operator method is based on the Lyapunov theory stability of solutions of ordinary differential equations systems. It is applicable to operator equations in Banach spaces, including in cases when the Frechet (Gateaux) derivative of a nonlinear operator is irreversible in a neighborhood of the initial value. In this paper, it is applied to the solution of the Dirichlet and Neumann problems for the Helmholtz equation and to determine the wave number in the inverse problem. The internal and external problems of Dirichlet and Neumann are considered. The Helmholtz equation is considered in domains with smooth and piecewise smooth boundaries. In the case when the Helmholtz equation is considered in domains with smooth boundaries, the existence and uniqueness of the solution follows from the classical potential theory. When solving the Helmholtz equation in domains with piecewise smooth boundaries, the Wiener regularization is carried out. The Dirichlet and Neumann problems for the Helmholtz equation are transformed by methods of potential theory into singular integral equations of the second kind and hypersingular integral equations of the first kind. For an approximate solution of singular and hypersingular integral equations, computational schemes of collocation and mechanical quadrature methods are constructed and substantiated. The features of the continuous method are illustrated with solving boundary problems for the Helmholtz equation. Approximate methods for reconstructing the wave number in the Helmholtz equation are considered.