A new recursive algorithm of stochastic approximation type based on the idea of averaging of trajectories is investigated. This type of algorithm, in contradiction to classical optimal algorithms, does not require any large amount of a priori information (e.g. the matrix of second derivatives at the zero point, etc.).
The asymptotic normality of the estimates as well as the proofs of the statements of almost sure convergence are given. It is also demonstrated that the proposed algorithm achieves the highest possible rate of convergence. The use of essentially new techniques in the proofs allows to substantially weaken the conditions of the theorems.
The case of linear problem is discussed first because the formulation of the results and proofs are the most clear for this case. Then a variety of classical optimization and identification problems is mentioned such as the nonlinear problem, stochastic optimization and, mainly, the estimation of the regression parameters.