The paper is devoted to the variational problem of the minimum for the functional \[ J(q(\cdot))= \int^T_0 [\dot x^2(t)+ (1-x(t)+ x^2(t))\dot y^2(t)+\dot z^2(t)]^{1\over2} dt \] on the set of curves \(q(\cdot)= (x(\cdot), y(\cdot), z(\cdot))': [0,T]\to \mathbb{R}^3\) under the following conditions: 1) \(q(\cdot)\) is absolutely continuous; 2) the nonholonomic constraint \(\dot z(t)= x^2(t)\dot y(t)\) holds: 3) \(q(0)= (0,-\ell,0)'\), \(q(T)=0\). This problem is equivalent to finding the shortest geodesic joining of two points \(q(0)\) and 0 in some sub-Riemannian structure \(A\in R^3\).
The author studies the possibility for approximation of the abnormal shortest geodesic of the sub-Riemannian structure \(A\) \[ x(t)\equiv 0, \quad y(t)\equiv t-\ell, \quad z(t)\equiv 0,\qquad t\in [0,\ell] \] by Riemannian geodesics. In doing that he uses the method of penalty functions which is justified by means of the solution to an auxiliary control problem with a parameter. Another result is concerned with establishing the nonregularity of sub-Riemannian structure \(A\) consisting in the loss of smoothness of the function of distance from the origin in the Carnot-Carathéodory metric defined by \(A\).