Integrating reinforcement learning, equilibrium points, and minimum variance to understand the development of reaching: a computational model.
Reaching and grasping objects is an evolutionarily late achievement. Throughout the animal kingdom, much object oriented action is achieved with the whole body, in mammals often with the head and snout. Because the vision sensor is anchored in the head, such object-oriented movements can be achieved with simple control strategies, such as visual servoeing (Ruf & Horaud, 1999), that make limited demands on perception, estimation, and movement planning. Reaching with an actuator that is separate from the main visual sensor is prevalent in primates. Humans excel at object oriented manipulation tasks, much exceeding the skills of other primates. This is a developmental achievement as witnessed by the long and intense period of learning to reach (von Hofsten, 1984, 1991; Thelen, Corbetta, & Spencer, 1996; Berthier & Keen, 2006) Spatial orientation and navigation may be achieved based on unsegmented visual information (Schöne, 1984; Hermer & Spelke, 1994). In contrast, reaching makes considerable demands on object perception. (1) To reach and grasp an object, that object must be visually segmented against the background and its pose be estimated. (2) Spatial information about the object must be transformed from a visual reference frame into a reference frame in which motor commands to the hands may be formed. Such coordinate transforms are computationally demanding (Pouget & Snyder, 2000; Schneegans, 2015) and learning them is a challenge (Chao, Lee, & Lee, 2010; Sandamirskaya & Conradt, 2013). (3) Motor commands must be generated that drive the hand toward the object and bring it into contact with the object with a small enough terminal velocity that enables grasping. This is particularly challenging as infants are weak relative to the mass of their limbs. Because the force/weight relationship changes during growth, the motor commands must be updated over development. (4) Generating and controlling successful reaches also entails solving the degree of freedom problem, that is, distributing to the many muscles that contribute to arm movement a