Abstract
Motor learning in the context of arm reaching movements has been frequently investigated using the paradigm of force-field learning. It has been recently shown that changes to somatosensory perception are likewise associated with motor learning. Changes in perceptual function may be the reason that when the perturbation is removed following motor learning, the hand trajectory does not return to a straight line path even after several dozen trials. To explain the computational mechanisms that produce these characteristics, we propose a motor control and learning scheme using a simplified two-link system in the horizontal plane: We represent learning as the adjustment of desired joint-angular trajectories so as to achieve the reference trajectory of the hand. The convergence of the actual hand movement to the reference trajectory is proved by using a Lyapunov-like lemma, and the result is confirmed using computer simulations. The model assumes that changes in the desired hand trajectory influence the perception of hand position and this in turn affects movement control. Our computer simulations support the idea that perceptual change may come as a result of adjustments to movement planning with motor learning.
Similar content being viewed by others
References
Aström K, Wittenmark B (1989) Adaptive control. Addison-Wesley, Boston
Bernier P, Burle B, Vidal F, Hasbroucq T, Blouin J (2009) Direct evidence for cortical suppression of somatosensory afferents during visuomotor adaptation. Cereb Cortex 19(9):2106–2113
Berret B, Chiovetto E, Nori F, Pozzo T (2011) Manifold reaching paradigm: how do we handle target redundancy? J Neurophysiol 106(4):2086–2102
Bhushan N, Shadmehr R (1999) Computational nature of human adaptive control during learning of reaching movements in force fields. Biol Cybern 81(1):39–60
Chien M-C, Huang A-C (2004) Adaptive impedance control of robot manipulators based on function approximation technique. Robotica 22(4):395–403
Darainy M, Mattar AAG, Ostry DJ (2009) Effects of human arm impedance on dynamics learning and generalization. J Neurophysiol 101(6):3158
Davidson P, Wolpert D (2004) Scaling down motor memories: de-adaptation after motor learning. Neurosci Lett 370(2):102–107
Flash T (1987) The control of hand equilibrium trajectories in multi-joint arm movements. Biol Cybern 57(4):257–274
Flash T, Hogan N (1985) The coordination of arm movements: an experimentally confirmed mathematical model. J Neurosci 5(7):1688
Galicki M (2007) Adaptive path-constrained control of a robotic manipulator in a task space. Robotica 25(1):103–112
Gomi H, Kawato M (1996) Equilibrium-point control hypothesis examined by measured arm stiffness during multijoint movement. Science 272(5258):117
Gribble PL, Ostry DJ, Sanguineti V, Laboissière R (1998) Are complex control signals required for human arm movement? J Neurophysiol 79(3):1409
Huang S, Tan K, Lee T (2008) Adaptive neural network algorithm for control design of rigid-link electrically driven robots. Neurocomputing 71(4–6):885–894
Ito S, Aoyama Y, Kawasaki H (2003) A static balance control under periodic external force. In: SICE 2003 annual conference, pp 600–605
Ito S, Kawasaki H (2005) Regularity in an environment produces an internal torque pattern for biped balance control. Biol Cybern 92(4):241–251
Izawa J, Rane T, Donchin O, Shadmehr R (2008) Motor adaptation as a process of reoptimization. J Neurosci 28(11):2883–2891
Kambara H, Kim K, Shin D, Sato M, Koike Y (2009) Learning and generation of goal-directed arm reaching from scratch. Neural Netw 22(4):348–361
Katayama M, Kawato M (1993) Virtual trajectory and stiffness ellipse during multijoint arm movement predicted by neural inverse models. Biol Cybern 69(5):353–362
Kistemaker DA, Soest AKJV, Bobbert MF (2007) Equilibrium point control cannot be refuted by experimental reconstruction of equilibrium point trajectories. J Neurophysiol 98(3):1075
Malfait N, Shiller D, Ostry D (2002) Transfer of motor learning across arm configurations. J Neurosci 22(22):9656–9660
Massone L, Bizzi E (1989) A neural network model for limb trajectory formation. Biol Cybern 61(6):417–425
Mitrovic D, Klanke S, Vijayakumar S (2011) Learning impedance control of antagonistic systems based on stochastic optimization principles. Int J Robotics Res 30(5):556
Miyamoto H, Kawato M, Setoyama T, Suzuki R (1988) Feedback-error-learning neural network for trajectory control of a robotic manipulator. Neural Netw 1(3):251–265
Mussa-Ivaldi F, Hogan N, Bizzi E (1985) Neural, mechanical, and geometric factors subserving arm posture in humans. J Neurosci 5(10):2732–2743
Nguyen-Tuong D, Peters J (2011) Model learning for robot control: a survey. Cogn Process 12(4):319–340
Ostry DJ, Darainy M, Mattar AAG, Wong J, Gribble P (2010) Somatosensory plasticity and motor learning. J Neurosci 30(15):5384
Shadmehr R, Mussa-Ivaldi FA (1994) Adaptive representation of dynamics during learning of a motor task. J Neurosci 14(5):3208
Slotine J-JE, Li W (1987) On the adaptive control of robot manipulators. Int J Robotics Res 6(3):49
Slotine J-JE, Li W (1991) Applied nonlinear control, vol 66. Prentice hall, Englewood Cliffs
Sutton R, Barto A (1998) Reinforcement learning: an introduction, vol 116. Cambridge University Press, Cambridge
Tee KP, Franklin DW, Kawato M, Milner TE, Burdet E (2010) Concurrent adaptation of force and impedance in the redundant muscle system. Biol Cybern 102(1):31–44
Todorov E, Jordan MI (2002) Optimal feedback control as a theory of motor coordination. Nat Neurosci 5(11):1226–1235
Todorov E, Jordan MI (1998) Smoothness maximization along a predefined path accurately predicts the speed profiles of complex arm movements. J Neurophysiol 80(2):696
Tsuji T, Morasso PG, Goto K, Ito K (1995) Human hand impedance characteristics during maintained posture. Biol Cybern 72(6):475–485
Uno Y, Kawato M, Suzuki R (1989) Formation and control of optimal trajectory in human multijoint arm movement. Biol Cybern 61(2):89–101
Wolpert DM, Ghahramani Z, Jordan MI (1995) An internal model for sensorimotor integration. Science 269(5232):1880
Acknowledgments
This research was supported by the National Institute of Child Health and Human Development (R01-HD075740) and by Le Fonds Quebecois de la Recherche sur la Nature et les Technologies (Quebec). Conflict of interest The authors declare that they have no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: 2-link system dynamics and kinematics
The 2-link dynamics of the arm in the horizontal plane are described as follows:
Here, \({M}(\varvec{\theta })\) and \(C(\varvec{\theta }, {\varvec{\dot{\theta }}})\) are given as follows:
As shown in Fig. 2, \(\theta _i\) denotes joint angle, \(L_i\) is link length, \(\ell _i\) is the distance to center of mass of the link, \(m_{i}\) is mass of the link, \(I_i\) is the moment of inertia about the center of mass, \(\tau _i\) denotes joint torque and \(i\) distinguishes the shoulder \((i=0)\) and the elbow \((i=1)\). The operation \(\dot{}\) denotes time derivative.
Note here that the left side of Eq. (39) can be described as the product of \({Y}_L\) and \(\varvec{\sigma }_L\) as shown in (7). In the case of 2-link arm dynamics, they are given as
Then, the following equation holds using \({\varvec{\hat{\sigma }}}_L\):
\({\hat{M}}(\varvec{\theta })\) and \({\hat{C}}(\varvec{\theta }, {\varvec{\dot{\theta }}})\), the estimates of \({M}(\varvec{\theta })\) and \({C}(\varvec{\theta },{\varvec{\dot{\theta }}})\), respectively.
On the other hand, the relationship between hand position and joint angles is given as follows:
Hand velocity are related to joint velocities by the Jacobian matrix \(J(\varvec{\theta }) \) as follows:
where
and
Appendix 2: Calculation of control law
Using (9), \({\varvec{\dot{\theta }}}_r\) becomes
where
Assuming that all the dynamical parameters are known, the control law (3) can be rewritten as follows:
It can be seen from Eq. (61) that the second term in Eq. (62) is a function of \(\varDelta \varvec{\theta }_d^*,\,\varDelta {\varvec{\dot{\theta }}}_d^*\), and \(\varDelta {\varvec{\ddot{\theta }}}_d^*\). If we compare Eq. (10) with Eq. (62), the second term of (10) will be written as follows using (13)
where
This gives us Eq. (16).
Appendix 3: Boundedness
Based on assumption A7, \(\varvec{\theta }_d^*,\,{\varvec{\dot{\theta }}}_d^*,\,{\varvec{\ddot{\theta }}}_d^*\) are bounded. \({ Y}_\psi (t)\) is also bounded as assumed in Theorem 1. Furthermore, \({ M}(\varvec{\theta })\) is bounded since each element contains only constants or cosine functions.
Because \(\dot{V} \le 0,\,V(0) \ge V > 0,\,V\) is bounded, i.e., \(\varvec{s}\) and \(\bar{\varvec{\sigma }}_c\) are bounded. The boundedness \(\varvec{s}\) ensures the boundedness of \({\varvec{\dot{\theta }}}\) and \({\varvec{\dot{\theta }}}_r^*\). The boundedness of \({\varvec{\dot{\theta }}}_r^*\) means that \(\varvec{\theta } \) is bounded (see Eq. (11)). Thus, \({ C}(\varvec{\theta }, {\varvec{\dot{\theta }}})\) is also bounded.
We can ensure the boundedness of \(\dot{\varvec{s}}\) because of Eq. (23) and the nonsingularity of \({M}(\varvec{\theta })\).
Because we can obtain the boundedness of \(\varvec{s}\) and \(\dot{\varvec{s}},\,\ddot{V}\) given by (29) becomes bounded.
Rights and permissions
About this article
Cite this article
Ito, S., Darainy, M., Sasaki, M. et al. Computational model of motor learning and perceptual change. Biol Cybern 107, 653–667 (2013). https://doi.org/10.1007/s00422-013-0565-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00422-013-0565-3