Abstract
When sharing load among multiple muscles, humans appear to select an optimal pattern of activation that minimizes costs such as the effort or variability of movement. How the nervous system achieves this behavior, however, is unknown. Here we show that contrary to predictions from optimal control theory, habitual muscle activation patterns are surprisingly robust to changes in limb biomechanics. We first developed a method to simulate joint forces in real time from electromyographic recordings of the wrist muscles. When the model was altered to simulate the effects of paralyzing a muscle, the subjects simply increased the recruitment of all muscles to accomplish the task, rather than recruiting only the useful muscles. When the model was altered to make the force output of one muscle unusually noisy, the subjects again persisted in recruiting all muscles rather than eliminating the noisy one. Such habitual coordination patterns were also unaffected by real modifications of biomechanics produced by selectively damaging a muscle without affecting sensory feedback. Subjects naturally use different patterns of muscle contraction to produce the same forces in different pronation–supination postures, but when the simulation was based on a posture different from the actual posture, the recruitment patterns tended to agree with the actual rather than the simulated posture. The results appear inconsistent with computation of motor programs by an optimal controller in the brain. Rather, the brain may learn and recall command programs that result in muscle coordination patterns generated by lower sensorimotor circuitry that are functionally “good-enough.”
Introduction
The human body has many more degrees of freedom than apparently needed to generate basic movements such as reaching or grasping. Resolution of this redundancy is a fundamental problem of biological motor control (Bernstein, 1967) that must also be addressed for the design of robotic and prosthetic systems that mimic or restore human movement (Loeb and Davoodi, 2005). The computational framework of optimal control theory has gained influence as a general theory of motor coordination because it can specify uniquely how behavioral goals should be achieved by minimizing costs such as the effort or variability of movement (Pedotti et al., 1978; Crowninshield and Brand, 1981; Davy and Audu, 1987; Loeb et al., 1990; Harris and Wolpert, 1998; Todorov and Jordan, 2002; Scott, 2004; Todorov, 2004; O'Sullivan et al., 2009; Diedrichsen et al., 2010). Although some aspects of natural motor behavior, such as typical patterns of muscle activity, are consistent with the output of optimal control models (Fagg et al., 2002; Haruno and Wolpert, 2005; Diedrichsen et al., 2010), how the nervous system generates this behavior is unknown. For instance, it is unclear whether the CNS achieves behavior that appears to be optimal through online, top-down optimization, or because the architecture of distributed sensorimotor networks has evolved to favor successful and efficient behavior.
Here we sought to resolve the issue of whether patterns of muscle coordination are “computed” by the brain via an online optimization process, or reflect habits developed when a central controller learns associations between tasks and strategies according to some performance criteria. An assertion of “optimal control” begs a cost-function, which for natural tasks is likely to be some unknown, weighted combination of terms related to speed, accuracy and effort. In the face of uncertainty about the exact cost to be minimized, it is difficult to determine whether an observed natural pattern of muscle coordination is actually optimal or simply reflects a good-enough local minimum that the organism has learned to use. An alternative approach to resolve the issue is to assess how the central controller responds to new conditions, since an online, optimal controller would be expected to alter muscle coordination patterns to match any changes in the properties of the musculo-skeletal plant. Conversely, habitual coordination patterns might tend to be retained when conditions change if learned associations between control strategies and tasks allow successful, albeit suboptimal, achievement of the behavioral goal. We therefore exposed human subjects to a range of experimental modifications of their muscle mechanics, and assessed whether muscle activation patterns were reoptimized according to the new properties of the limb. In contrast to predictions from optimal control, habitual patterns of muscle coordination were preserved in the face of all biomechanical manipulations tested, at least over hours and hundreds of trials as tested in these experiments.
Materials and Methods
Subjects.
Fifteen healthy, right-handed volunteers (age, 22–38; 14 males, 1 female), including two of the authors, participated in the study. When subjects participated in more than one experiment (n = 5), testing sessions were separated by at least 3 weeks. All provided written consent to the procedures, which conformed to the declaration of Helsinki and were approved by a local ethics committee.
General procedure.
Subjects moved a cursor from the center of a two-dimensional display to 16 targets distributed symmetrically around the origin (i.e., 22.5° apart). Under baseline conditions, the cursor position represented the force exerted at the wrist joint such that 22.5 N was required to achieve the target. In some experimental conditions, the cursor position represented simulated force that was reconstructed from real-time muscle activity as indicated below (i.e., “EMG-driven” conditions). Participants were asked to reach the target with a movement time of between 150 and 250 ms, and to hold the cursor for 1 s on target (either a trapezoid ±8° from target direction by 10% of radial distance to target, or a circle of radius 10% of target distance). A high-pitched tone signaled target acquisition and a low-pitched tone indicated trial end if the target was not acquired within 2 s. Feedback on movement time was given after each trial.
Apparatus.
A 6 df force/torque transducer (JR3 Technologies, 45E15A-I63-A 400N60S) coupled with a custom manipulandum, similar to that described previously (de Rugy and Carroll, 2010; Selvanayagam et al., 2011), registered flexion/extension and abduction/adduction forces respectively to the x and y display axes. The wrist was fixed by an array of adjustable supports contoured to fit the hand at the metacarpal-phalangeal joints (12 contacts) and the wrist just proximal to the radial head (10 contacts).
EMG.
Electromyographic signals were recorded from Extensor Carpi Ulnaris (ECU), Extensor Carpi Radialis brevis (ECRb) and longus (ECRl), Flexor Carpi Radialis (FCR), Flexor Carpi Ulnaris (FCU), and first dorsal interosseous (FDI, Experiment 4 only), either with fine-wire intramuscular electrodes (75 μm diameter, 2 mm stripped from insulation for recording sites, single wires inserted at 1.5 cm interelectrode distance) or self-adhesive surface electrodes (3 subjects in Experiment 1, all subjects in Experiment 3, 12-mm-diameter recording surface; 2 cm interelectrode distance). Signals were bandpass filtered from 30 Hz to 1 kHz, amplified 200–5000 times (Grass P511, Grass Instruments, AstroMed) and sampled at 2 kHz. Electrode locations were determined according to procedures previously reported (Selvanayagam et al., 2011).
Force reconstruction.
Tuning curves for each muscle were constructed from the mean rectified EMG during the hold-phase of the task in five trials to each target (6 consecutive trials to each target, first trial discarded, target direction randomly ordered; Fig. 1b). This activity was normalized by the maximal EMG obtained during maximal voluntary contraction in any of the 16 target directions. The magnitude and direction of the virtual “pulling vector” for each muscle that resulted in the best aiming performance when combined with the actual muscle activity was determined via an optimization procedure (see next section on the determination of the “virtual pulling vector”). These pulling vectors likely differed from the “true” anatomical pulling vectors of the five muscles, both because we neglected the contributions of finger flexors and extensors, and because EMG does not provide a perfect representation of muscle activity. Despite this, when the pulling vectors were multiplied by the rectified and low-pass filtered (1.3–2 Hz) EMG of the five muscles online, the reconstructed “force” provided a close spatiotemporal match to the true force exerted at the wrist (Fig. 1e). In each experiment, a set of trials (6 consecutive trials to each target) was performed with the cursor driven online by EMG to confirm similarity in performance and muscle activities with baseline (i.e., between EMG-driven and “force-driven” modes).
Determination of the virtual pulling vector of muscles.
The virtual pulling vector of muscles was specified as the set of vectors that resulted in the best aiming performance when combined with the actual muscle activity. This set of vectors was determined by a custom gradient descent algorithm based on a simple quadratic cost function that penalizes errors between the targets and the reconstructed reaches. The steps in the gradient descent are as follows: (1) Assign random values to the initial set of pulling vectors in the physiological range of muscle force and direction; (2) Pick a muscle at random and modify its pulling vector by changing its endpoint by a step in four orthogonal directions. The cost associated with each of the 5 pulling vectors (i.e., the original and the four modified for that muscle) was then calculated as the summed squared error between targets and reconstructed reaches. The pulling vector that produced the lowest cost was retained; (3) One iteration of the model was said to be completed when each muscle had been optimized once; (4) The whole model was iterated multiple times until the overall cost converged to a low value.
Experiment 1—Simulated loss of one muscle.
For six subjects, the pulling vector for the ECRl muscle was set to 0, such that activation of the muscle had no effect on the cursor. ECRl was chosen for this manipulation because the angle between the pulling vectors of its two neighboring muscles is smallest (compare Fig. 1c). In principle, the simulated loss of the mechanical action of this muscle should therefore be more easily compensated using its closest neighbors. Three sets of 160 trials were performed (10 trials to each target, order randomized within each block). Tuning curves were established by a set of six trials to each target after each set of 160. Thus, by the end of the final set, participants had completed 768 trials with the altered “virtual” biomechanics.
Experiment 2—Simulated increase in motor noise of one muscle.
Random values were drawn from a Gaussian distribution, rectified and low-pass filtered at 0.8 Hz, multiplied by the normalized amplitude of the instantaneous, filtered EMG amplitude (and an additional scaling factor to ensure a large behavioral effect), and added to the EMG signal. Noise was added to ECRl (n = 3) or FCU (n = 3), and a protocol similar to that of Experiment 1 was conducted with this altered condition.
Experiment 3—Actual damage to one muscle.
FCU was electrically stimulated to produce involuntary force while it was forcibly lengthened. FCU was chosen for this manipulation because it could most easily be selectively activated using transcutaneous nerve stimulation. Two second trains of supramaximal electrical pulses were delivered to the ulnar nerve at the wrist (0.2 ms duration, 50–40 Hz, 9–25 mA), while the joint was forcibly abducted via a pulley and weight system described previously (Carroll et al., 2009). Relatively low currents (8–25 mA) were required to activate FCU maximally because of the superficial location of the ulnar nerve at the elbow. This meant that the discomfort associated with the electrical stimulation was minimal. Exercise-induced muscle damage can lead to large, temporary force decrements, but full recovery of force occurs within 2–3 weeks (Clarkson et al., 1992). The supramaximality of the stimulus was checked regularly by monitoring the amplitude of compound muscle action potentials (CMAPs) and intensity was increased when required to account for activity-dependent changes in axonal excitability (Burke and Gandevia, 1999). This produced selective and long-lasting muscle damage to the FCU, as indicated by stable reductions in MVC force in flexion and ulnar deviation, twitch amplitude induced by a single supramaximal stimulus, and evoked force responses to 15 Hz and 100 Hz pulse trains immediately after the damage and at the end of the experiment (i.e., after ∼2 h task performance). The degree of exercise-induced muscle damage is typically greater in large diameter, fast muscle fibers (Jones et al., 1986; Vijayan et al., 2001), and fast fibers make a disproportionately large contribution to total muscle force produced both during MVC and electrically evoked contractions. Because of this, the size of the force reduction in the muscle fibers contributing to the target aiming task was estimated by reoptimizing the length of the initially determined pulling vector in the damaged muscle on the basis of the EMG produced during task performance immediately after the damaging exercise. The estimated force reduction varied between 15.0 and 71.6% of baseline, and was linearly related to the reduction in electrically evoked force (r2 = 0.63, p = 0.035). A protocol similar to that of Experiment 1 and 2 was conducted with the damaged muscle.
Experiment 4—Simulated effects of postural change.
Virtual pulling vectors were extracted from task performance (as above) with the wrist supinated 80° from neutral and were applied during task performance in EMG-driven mode with the wrist pronated 80° from neutral. MVC and both force- and EMG-driven baseline sets were performed in supination and pronation, before three long (random target order) and three short (consecutive to each target) sets were performed with the altered virtual biomechanics as above.
Data analysis.
Muscle activity differences from before to after modification of the biomechanics were tested using the Wilcoxon signed-rank test, and effects of practice over the acquisition blocks were tested using Friedman's ANOVAs. The Wilcoxon signed-rank test was used to compare the actual increase in FCU activity with that predicted by optimal control (Experiment 3), and to test δ against 0 and differences among the post-blocks (Experiment 4). All reported values are means ± SD.
In Experiments 1 and 3, linear regressions were used between muscle activity in condition of altered biomechanics and muscle activity predicted by a proportional scaling of the baseline activity patterns. Specifically, linear regressions were used to describe the relationship between the difference in the amplitude of muscle activities summed over all target directions under baseline and modified biomechanics (Mpost/Mpre), versus the differences in activity under baseline conditions and the activity predicted should the same relative activation of muscles for each force direction at baseline be used under the altered biomechanics (Msame/Mpre) (see next section on calculation of Msame). In Experiment 3, linear regression was calculated for the relationship between Mpost/Mpre and the difference in muscle activity predicted by an optimal control model before and after muscle damage (Mopt.post/Mopt.pre). Mopt.pre and Mopt.post were determined using an optimization procedure described previously that achieves targets while minimizing the sum of squared muscle activities (Fagg et al., 2002). This optimization procedure was conducted on the pulling vectors initially extracted from baseline muscle activity to obtain Mpre, and on the pulling vectors reoptimized after muscle damage to obtain Mpost.
In Experiment 4, the summed muscle activity obtained in altered biomechanics (Mpost) was compared both to the activity obtained at baseline in supination (Msup) and to the activity predicted should the muscles be activated in the same proportions as at baseline in pronation (Msame, determined as in Experiment 1) within a single measure: δ = (dif(Mpost,Msame) − dif(Mpost,Msup))/dif(Msame,Msup). The measure dif(condition1, condition2) was defined as the sum for all muscles of the absolute value of the signed difference in muscle activity between the two conditions summed over the 16 target directions. This measure is such that δ = 1 when Mpost = Msup, and δ = −1 when Mpost = Msame.
Calculation of Msame.
In Experiment 1, Msame was calculated using the direction and extent of the actual force produced to achieve the targets under altered biomechanics: The relative contributions of muscles were determined from interpolation of the muscle tuning curves at baseline for the direction of the actual force produced, and the magnitude of muscle activity was obtained by scaling these contributions to the extent of that force. In Experiment 3, Msame was calculated using the direction and extent of the forces simulated by combining EMGs at baseline interpolated for multiple directions (256) with the novel pulling vector reoptimized after the muscle had been damaged: The relative contributions of muscles were determined from the interpolated muscle tuning curves at the simulated forces that best matched the target directions, and the magnitude of muscle activity was obtained by scaling these contributions to the extent of that force.
Results
We tested predictions from optimal control theory in response to both real and virtual manipulation of the mechanical action of wrist muscles. We used computer models to convert EMG recordings into direction and magnitude of force produced by the five major muscles that generate force at the wrist when the fingers are free in space (Fig. 1). These virtual pulling vectors were then combined to produce a real-time reconstruction of the two-dimensional force that provided a close spatiotemporal match to measured force at the hand (Fig. 1e). The task performed in all experiments required participants to move a cursor from the center of a two-dimensional display and to hold the cursor for 1 s on each of 16 targets distributed symmetrically about the origin. The cursor position represented either the resultant force exerted at the wrist joint (in force-driven conditions), or the reconstructed force specified by the virtual biomechanics and the real-time muscle activity (in EMG-driven conditions). Figure 1f illustrates that the muscle tuning curves obtained in these two conditions are similar.
Adaptation to novel virtual biomechanics with muscle loss—Experiment 1
An extensor muscle was virtually cut, such that activation of the muscle had no effect on the EMG-driven cursor. As expected, subjects were able to acquire targets near to the pulling direction of the missing muscle by increasing the activation of the two neighboring muscles (Fig. 2). The interesting question is how the CNS treats the muscle that no longer has an effect on the cursor. An unrestricted optimal controller with energy expenditure in the cost function would reduce, and eventually abolish, the activity of the irrelevant muscle. However, we found that the activity of the missing muscle was dramatically increased (i.e., multiplied by 2.87 ± 0.56 from pre- to post-block, T = 0, p = 0.028), and that there was no reduction from this level even after subjects had completed 768 trials (∼1.5 h) under the new conditions (χ2(2) = 2.33, p = 0.31; Fig. 2b,c). The magnitude of muscle activity in all muscles under the new virtual biomechanics was well described by a linear scaling of the muscle activity obtained before the altered biomechanics at baseline, according to the magnitude and direction of the actual force produced at the wrist under virtual conditions (r2 = 0.85, p < 0.001; Fig. 2d). It is important to note that target acquisition under the simulated biomechanics required substantial deviations in both direction and magnitude of the actual wrist force produced relative to baseline conditions. This indicates that the task was resolved by modifying the wrist force, but not the muscle activity pattern habitually used to produce that force.
Adaptation to novel virtual biomechanics with motor noise—Experiment 2
Variability in force production (“motor noise”) has also been hypothesized as a cost to optimize (Harris and Wolpert, 1998; Haruno and Wolpert, 2005; O'Sullivan et al., 2009; Diedrichsen et al., 2010) but it covaries with muscle activity under natural conditions (Jones et al., 2002). To dissociate these factors, we manipulated the virtual biomechanics to increase the signal-dependent noise associated with muscle activation in one of the five muscles (i.e., an extensor muscle, n = 3; or a flexor muscle, n = 3). Random values drawn from a zero-mean Gaussian distribution with SD proportional to the current level of muscle activation were low-pass filtered (0.8 Hz), and added to the EMG signal of the selected muscle. This caused a dramatic increase in cursor variability toward targets close to the pulling direction of the noisy muscle (Fig. 3b), directly interfering with task performance. The response of an unrestricted optimal controller to this perturbation would be to reduce activation of the noisy muscle and produce a compensatory increase in the activation of the two more stable neighboring muscles. Instead, there was no change in the activation level of the muscle with exaggerated signal-dependent noise, even after 768 trials (χ2(3) = 3.80, p = 0.28; Fig. 3c,d), and only a slight increase (< 3%) for the neighboring muscles in the last block (χ2(3) = 10.6, p = 0.014).
Adaptation to actual damage of one muscle—Experiment 3
Because optimal controllers require an accurate estimate of the state of the system (Scott, 2004; Todorov, 2004), it is possible that the failure of the CNS to reoptimize muscle activation according to the new virtual biomechanics relates to discrepancies between visual information about cursor position (i.e., reconstructed force) and proprioceptive information from receptors in the limb (e.g., real force sensed by mechanoreceptors). Therefore, we tested whether reoptimization of muscle activation patterns occurs after a modification of the actual mechanical actions of a single muscle that preserves the integrity of sensory feedback. We induced selective and long-lasting muscle damage in one of the flexor muscles by forcibly lengthening the muscle during involuntary activation produced by supramaximal electrical stimulation of the relevant peripheral nerve. This produced temporary, but stable reductions in the relationship between EMG and force, as estimated by recomputing pulling vectors during task performance after the damaging exercise (see Materials and Methods, Experiment 3). Exercise-induced muscle damage does not, however, affect muscle somatosensors (Gregory et al., 2002, 2004). We compared the changes in muscle activation after the damaging exercise with the changes predicted from an optimal control model (Fagg et al., 2002) according to the new force-generating capacity of the muscles. As in Experiment 1, the muscle activation was well predicted according to a rescaling of original patterns of activity (r2 = 0.87, p < 0.001; Fig. 4d). In contrast, the post-damage activity was not related to the change predicted by optimal control (r2 = 0.06, p = 0.14; Fig. 4e). The size of the increase in activity in the damaged muscle (i.e., multiplied by 1.80 ± 0.44 from pre- to post-block, T = 0, p = 0.018) was greater than the amount predicted by optimal control in all subjects (1.17 ± 0.14; T = 0, p = 0.018), and did not change with extended exposure to the task (χ2(2) = 4.57, p = 0.10; Fig. 4f). Thus, participants did not reoptimize their muscle activation patterns despite congruent sensory feedback from vision and proprioception.
Adaptation to simulated postural change—Experiment 4
The failure of the CNS to reoptimize muscle activation in the conditions above might relate to an inability to produce the patterns of muscle activity specified by optimal control due to constraints imposed by the structure of the neuro-motor system. To address this issue, we introduced a more natural perturbation in Experiment 4 by simulating the biomechanics of a posture different from that in which the subjects performed the task. The tuning curves and mechanical actions of the wrist muscles are known to change with forearm rotation (Kakei et al., 1999). We exploited this feature by applying the pulling vectors extracted from the wrist during task performance in full supination to the EMG recorded online during task performance with the wrist fully pronated. Assuming near-optimal behavior at baseline in the different postures, we can therefore be sure that the nervous system is capable of producing optimal muscle activations when the posture was virtually changed, even if the structure of the neural circuits prevents the production of some activity patterns. Once again, however, muscle activation patterns were closer to those produced at baseline in the actual wrist position than at baseline in the virtual position for all subjects (δ below zero for all post-blocks; T = 0, p = 0.043; with an effect of time; χ2(2) = 4.57, p = 0.041; Fig. 5b).
Discussion
What is “optimal” and how might it be computed?
It is important to distinguish between iterative learning algorithms that gradually minimize a cost function but may become stuck in good-enough local minima, and analytical algorithms that actually compute a globally optimal strategy, presumably by using an internal model of the system to be optimized. Because the actual cost function being used by the organism cannot be known, it will never be possible to look at a given behavior and state whether or not it is globally optimal. For that reason, we designed the experiments presented here to change the properties of the musculoskeletal plant in ways that should force the nervous system to adopt qualitatively different motor programs to maintain global optimality regardless of the cost function.
The data show that contrary to predictions from optimal control theory, motor coordination is not continuously optimized at the level of individual muscles. Instead, habitual muscle coordination is surprisingly robust to various real and virtual manipulations of the limb's biomechanics. Furthermore, the finding that muscle coordination is tied to the actual limb posture suggests that good-enough solutions to muscle redundancy are generated by a hierarchical control scheme in which muscle activity is partly determined by low-level networks whose regulatory functions are shaped by sensory feedback representing the current posture. In this type of scheme, control processes are distributed over sensorimotor networks at multiple levels, with high-order levels acting as controllers for lower levels, and low-order levels acting as regulators for higher levels (Raphael et al., 2010). The processes of evolution, development, and adaptation determine in parallel both the structure of the circuitry within each level and the characteristics of the musculoskeletal system. Even low-order levels of the hierarchical control scheme should therefore carry useful information about the nature of the plant and the set of likely tasks to be encountered, such that explicit specification of muscle activations from the brain may not be required for successful task performance in most conditions.
Under natural conditions, however, it is difficult to dissociate online, top-down control of muscle activation from control that is embedded in the properties of a hierarchical control system because both may produce outcomes that are “near-optimal” for sensible cost functions. To address this issue, we exposed human subjects to a range of simulated and actual modifications to their limb biomechanics. Simulated loss of one muscle in Experiment 1 was resolved by simply modifying the direction and magnitude of the wrist force instead of changing the muscle activity pattern habitually used to produce that force, even though this implied a dramatic increase in the activity of the lost muscle that compromised energetic efficiency. Similarly, muscle activity habitually used to produce force was maintained when motor noise was selectively added to one muscle in Experiment 2, thereby compromising the stability of the simulated force output and task success. Results from these two experiments therefore argue against online reoptimization of muscle activity based on either of the two major cost functions (i.e., effort and variability) that have been proposed to resolve muscle redundancy using optimal control (Fagg et al., 2002; Haruno and Wolpert, 2005; Diedrichsen et al., 2010). It is not clear whether optimization on the basis of some alternate cost function might be consistent with the data, but it would appear unlikely that any viable function could be heavily weighted toward costs associated with the physical state of the plant.
Does the brain maintain and use a currently accurate internal model of the musculoskeletal plant?
The results of Experiment 3, in which the actual force generation capacity of a muscle was reduced, demonstrate that the failure of the nervous system to reoptimize muscle activation according to the simulated biomechanics was not due merely to discrepancies between visual information about cursor position and proprioceptive information from receptors in the limb. Because the selective muscle damage used to achieve the force reduction should not have affected the function of mechanoreceptors in the limb (Gregory et al., 2002, 2004), accurate sensory information on which to update any internal model or other state estimation required for computation of an optimal control strategy (Scott, 2004; Todorov, 2004) should have been available. Thus, an optimal controller minimizing energy cost should have reduced the activity of the damaged muscle relative to that of its more efficient undamaged neighbor. As in Experiment 1, however, the muscle activation was well described by a simple linear rescaling of original patterns of activity, and not reoptimized for the novel biomechanics. It should be noted that recruitment of the virtually cut muscle in Experiment 1 and the actually damaged muscle in Experiment 3 would be associated with substantial energy consumption related to activation (Tsianos et al., 2012), so the observed increase in recruitment is not consistent with minimization of energetic cost. It is possible that the brain has an internal model of the defective muscle but chooses not to use it to optimize energy consumption or that it is unable to make corrections to that internal model within the time frame of these experiments. One advantage of an internal model is that it can be used to compute solutions to problems more rapidly than they can be discovered by trial-and-error learning, but this begs the question of how rapidly the model itself can be updated. A persistently inadequate internal model could be a source of persistently suboptimal motor habits.
Is motor performance limited by an impoverished set of hard-wired synergies?
An alternate view of how animals deal with muscle redundancy is that the CNS uses stereotyped combinations of muscles as “synergies” to reduce the dimensionality of the control problem (Bizzi et al., 2000; d'Avella et al., 2006). According to this perspective, muscle synergies might not be sufficiently flexible to enable selective changes in muscle activations needed to respond optimally to our conditions in Experiments 1–3. In this case, an optimal controller could still operate on muscle synergies instead of individual muscles. Indeed, in recent theoretical work optimal controllers have been combined with lower dimensional muscle synergies to simplify the control problem (Lockhart and Ting, 2007; Berniker et al., 2009). Similarly, collateral projections of cortico-motoneurons to multiple wrist motoneuron pools (Fetz et al., 1989) might prevent independent activation of muscles that are functionally coupled. If the nervous system is actually capable of producing the necessary muscle activations in a different but natural posture as we observed in Experiment 4, then this limitation would seem not to obtain.
One potential criticism of the logic above is that the coordination patterns required in an alternate posture might be difficult to produce due to the influence of afferent feedback about limb position on the resting state of motor circuits. However, if an optimal controller were unable to modify afferent feedback gains as needed to allow replication of coordination patterns typically produced in an alternate posture, it could hardly be claimed that such a controller computed and specified any muscle activity patterns online, let alone optimal ones. The behavior actually observed suggests that the central controller learns associations between tasks and control strategies that are locally optimal in the sense that they tend to increase performance and/or reduce cost for regularly encountered conditions, but that these associations then become motor habits that are not readily changed. Such a phenomenon would explain the emergence of synergies themselves, as learned rather than hard-wired patterns of muscle use. The fact that various subjects exhibit similar synergies would then reflect similarities in their biomechanical constraints and in the circumstances in which they learned such habits.
Does optimal control operate over longer time periods than adaptation?
The conclusions drawn from this study are based on subject responses during the early stages of exposure to altered biomechanics. Subjects practiced under the changed conditions for ∼1.5–2 h, and completed between 750 and 800 movements in this period. Successful completion of the tasks under the imposed virtual or real changes in musculoskeletal function required substantial changes to motor programs, which appeared to be learned rapidly. Approximately half of these movements involved target directions for which substantial activation of the modified muscle was observed at baseline. In the course of practicing all possible movement directions, subjects would have obtained a complete set of information about the actual, current properties of both modified and neighboring muscles. This would presumably be required for controllers based on updatable internal models of the musculoskeletal plant. The argument could be made that our training was not lengthy enough to allow reoptimization, but this raises the question of why the control system was able to modify control signals to achieve the task within the timeframe of our experiments, but not to perform the task in an optimal way. Optimization that minimizes effort and motor variability can explain natural patterns of muscle activity, but the nervous system appears not to reoptimize such patterns over timescales that are typical of sensorimotor adaptation (Shadmehr et al., 2010).
It has long been known that permanent changes in the innervation or mechanical action of muscles used for locomotion lead to compensatory changes in their recruitment that are slow to develop and incomplete at best (Sperry, 1945; Forssberg and Svartengren, 1983; O'Donovan et al., 1985; Loeb, 1999; Slawinska and Kasicki, 2002). Muscles that are mechanically suitable for surgical tendon transfers in injured hands differ substantially in the ability of the brain to relearn their function (Fridén, 2005). These are extreme forms of the types of changes used in the present experiments that force us to consider how a theory of optimal control can be tested experimentally. If a body of experimental data is used to infer or even compute performance criteria (Terekhov and Zatsiorsky, 2011), those same data cannot be used to test the hypothesis that performance is optimal. Changing the musculoskeletal system as was done in these experiments provides an opportunity for such tests that, in the event, consistently failed. It is not clear what constraints on extent or rate of optimization must now be added to maintain a viable theory.
Are motor programs computed by optimization or learned and recalled?
Our results argue against the view that muscle coordination is specified online by optimal control. This raises the broader question of whether motor commands of any sort (i.e., those representing higher order features of movement) are computed by optimization in the brain or emerge from recalled control strategies operating on a hierarchical control system (Loeb et al., 1999; Raphael et al., 2010). Although the outcomes of both processes might often be well described by optimal control models, they have very different implications for the nature of processing performed by the brain, and therefore for applied fields such as neural prosthetics, brain-machine interfaces, telerobots and rehabilitation. Is the massive circuitry and processing power of the human brain devoted to computation of globally optimal solutions, or to classification, recall, and generalization of good-enough solutions? If the argument for computation is based on optimal control, then the evidence is mixed at best. Task goals are achieved in a near-optimal way in response to some types of perturbations (Chhabra and Jacobs, 2006; Izawa et al., 2008) but not to others (Ganesh et al., 2010; Kistemaker et al., 2010).
Recent theoretical work demonstrated the capabilities of a hierarchical control scheme to simplify both motor execution, by acting on a low-level controller that judiciously augments the dynamics of the plant (Todorov et al., 2005), and motor learning, by exploiting the multiple solutions that arise while randomly adjusting the gains of a complex but highly evolved spinal network (Raphael et al., 2010). Such hierarchical systems would account for the persistence of habitual coordination patterns that remained clearly suboptimal to all of the perturbations tested in this study.
Footnotes
This research was supported by a grant from the Australian Research Council (ARC DP0770982) to A.d.R. and T.J.C. We thank D.M. Wolpert, C. Ghez, J.C. Rothwell, and J.M. Donelan for feedback on an earlier version of the manuscript, and D. Lloyd for the figures of the apparatus.
- Correspondence should be addressed to Aymar de Rugy, Centre for Sensorimotor Neuroscience, School of Human Movement Studies, Room 424, Building 26, University of Queensland, St Lucia QLD, 4072, Australia. aymar{at}hms.uq.edu.au