Neurodynamics and nonlinear integrable systems of Lax type. (English) Zbl 0822.92003
We consider neurodynamics as a new topic of applied analysis of nonlinear integrable systems. We here take the position that we regard the problem of learning as a challenging problem in nonlinear dynamical systems. First we discuss a stochastic learning equation of Hebb type with a nonlinear damping term. Averaging the learning equation we derive a nonlinear dynamical system which is a gradient system for a potential function. Stable equilibrium points of the averaged learning equation are discussed. It is shown that the averaged learning equation also can be regarded as a constrained harmonic motion having a Lax pair representation. A competition learning rule appears as a first integral of the dynamical system. A relationship to the finite nonperiodic Toda lattice equation which is a typical nonlinear integrable system is discussed. It is also shown that a Lax type equation also emerges in the Hopfield model.
MSC:
| 92B20 | Neural networks for/in biological studies, artificial life and related topics |
| 37J35 | Completely integrable finite-dimensional Hamiltonian systems, integration methods, integrability tests |
| 37K10 | Completely integrable infinite-dimensional Hamiltonian and Lagrangian systems, integration methods, integrability tests, integrable hierarchies (KdV, KP, Toda, etc.) |
| 60H30 | Applications of stochastic analysis (to PDEs, etc.) |
| 68T05 | Learning and adaptive systems in artificial intelligence |
Keywords:
neurodynamics; nonlinear integrable systems; stochastic learning equation of Hebb type; gradient system; averaged learning equation; constrained harmonic motion; Lax pair representation; finite nonperiodic Toda lattice equation; Lax type equation; Hopfield modelReferences:
| [1] | S. Amari, Neural theory of association and concept-formation. Biol. Cybern.,26 (1977), 175–186. · Zbl 0364.92016 · doi:10.1007/BF00365229 |
| [2] | S. Amari and A. Takeuchi, Mathematical theory on formation of category detecting nerve cells. Biol. Cybern.,29 (1978), 127–136. · Zbl 0383.92003 · doi:10.1007/BF00337348 |
| [3] | P. Baldi and K. Hornik, Neural networks and principal component analysis: learning from examples without local minima. Neural Networks,2 (1989), 53–58. · doi:10.1016/0893-6080(89)90014-2 |
| [4] | A.M. Bloch, A completely integrable Hamiltonian system associated with line fitting in complex vector spaces. Bull. Amer. Math. Soc., (New Ser.)12 (1985), 250–254. · Zbl 0604.58032 · doi:10.1090/S0273-0979-1985-15365-0 |
| [5] | R.W. Brockett, Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems. Linear Algebra Appl.,146 (1991), 79–91; also Proc. 27th IEEE Conf. on Decision and Control, 1988, 799–803. · Zbl 0719.90045 · doi:10.1016/0024-3795(91)90021-N |
| [6] | P. Deift, F. Lund and E. Trubowitz, Nonlinear wave equations and constrained harmonic motion. Comm. Math. Phys.,74 (1980), 141–188. · Zbl 0435.35072 · doi:10.1007/BF01197756 |
| [7] | P. Deift, T. Nanda and C. Tomei, Ordinary differential equations and the symmetric eigenvalue problem. SIAM J. Numer. Anal.,20 (1983), 1–22. · Zbl 0526.65032 · doi:10.1137/0720001 |
| [8] | S. Geman, Some averaging and stability results for random differential equations. SIAM J. Appl. Math.,36 (1979), 86–105. · Zbl 0412.60069 · doi:10.1137/0136009 |
| [9] | J.J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci. USA,81 (1984), 3088–3092. · Zbl 1371.92015 · doi:10.1073/pnas.81.10.3088 |
| [10] | P.S. Krishnaprasad, Symplectic mechanics and rational functions. Ricerche Automat.,10 (1979), 107–135. |
| [11] | H. Kushner and D. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems. Appl. Math. Sci., Vol. 26, Springer-Verlag, New York, 1978. · Zbl 0381.60004 |
| [12] | J. Moser, Finitely many mass points on the line under the influence of an exponential potential – An integrable system. Dynamical Systems, Theory and Applications (ed. J. Moser), Lecture Notes in Phys., Vol. 38, Springer-Verlag, New York, 1975, 467–497. · Zbl 0323.70012 |
| [13] | Y. Nakamura, A new approach to the linear prediction problem for infinite stationary processes in terms of Riemann-Hilbert transformation theory. IMA J. Math. Control Inform.,6 (1989), 347–358. · Zbl 0699.60029 · doi:10.1093/imamci/6.3.347 |
| [14] | Y. Nakamura, Geometry of rational functions and nonlinear integrable systems. SIAM J. Math. Anal.,22 (1991), 1744–1754. · Zbl 0738.93018 · doi:10.1137/0522108 |
| [15] | Y. Nakamura, Lax equations associated with a least squares problem and compact Lie algebras. Recent Developments in Differential Geometry (ed. K. Shiohama), Adv. Stud. Pure Math., Vol. 22, Math. Soc. Japan, Tokyo, 1993, 213–229. · Zbl 0799.58031 |
| [16] | Y. Nakamura, A new dynamical system that leads to eigenvalues. Japan J. Indust. Appl. Math.,9 (1992), 133–139. · Zbl 0757.58020 · doi:10.1007/BF03167198 |
| [17] | Y. Nakamura, Completely integrable gradient systems on the manifolds of Gaussian and multinomial distributions. Japan J. Indust. Appl. Math.,10 (1993), 179–189. · Zbl 0814.58021 · doi:10.1007/BF03167571 |
| [18] | Y. Nakamura, Gradient systems associated with probability distributions. Japan J. Indust. Appl. Math.,11 (1994), 21–30. · Zbl 0811.58036 · doi:10.1007/BF03167211 |
| [19] | Y. Nakamura, Lax pair and fixed points analysis of Karmarkar’s projective scaling trajectory for linear programming. Japan J. Indust. Appl. Math.,11 (1994), 1–9. · Zbl 0799.90088 · doi:10.1007/BF03167209 |
| [20] | Y. Nakamura, Developments of applied analysis based upon nonlinear integrable systems (in Japanese). Bull. Japan Soc. Indust. Appl. Math.,2 (1992), 330–342. |
| [21] | E. Oja, A simplified neuron model as a principal component analyzer. J. Math. Biol.,15 (1982), 267–273. · Zbl 0488.92012 · doi:10.1007/BF00275687 |
| [22] | E. Oja and J. Karhunen, On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix. J. Math. Anal. Appl.,106 (1985), 69–84. · Zbl 0583.62077 · doi:10.1016/0022-247X(85)90131-3 |
| [23] | J. Rubner and K. Schulten, Development of feature detectors by self-organization, A network model. Biol. Cybernet.,62 (1990). 193–199. · doi:10.1007/BF00198094 |
| [24] | W.W. Symes, TheQR algorithm and scattering for the finite nonperiodic Toda lattice. Physica,4D (1982), 275–280. · Zbl 1194.37112 |
| [25] | M. Toda, Studies of a non-linear lattice. Phys. Rep.,8 (1975), 1–125. · Zbl 0298.55009 · doi:10.1016/0370-1573(75)90018-6 |
| [26] | C. von der Malsburg, Self-organization of orientation sensitive cells in the striate cortex. Kybernetika,14 (1973), 85–100. · doi:10.1007/BF00288907 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
