Motor sequences, Reinforcement Learning and the Brain

June 7, 2013, 1PM

인문신양학술관(4) 3 국제회의실



People often execute a quite complicated motor sequences effortlessly like driving a car once it is learned. Studies of several machine learning theories have been applied to processes of learning in motor sequence. One of most promising theories is the reinforcement learning (RL) in which an agent can choose optimal sequences of actions or states by maximizing accumulative rewards. Recently neural components for RL have been reported so that human and animals likely adapt RL-like learning. In this talk, I will present two experimental results showing neural substrate bridging between motor sequence and reinforcement learning. In the first experiment, I recorded neuronal activities from SMA (Supplementary Motor Area) and pre-SMA of two rhesus monkeys brains while the monkeys were executing motor sequences by moving joysticks. In this task, the number of movements for the monkey to get reward varies from trial to trial. Therefore, values in RL context (how close to reward) were dissociated from an ordinal position in a sequence for the same directional movements. The result showed that neurons in both the areas modulate their activities more for the temporal proximity to a reward (i.e. value) than the ordinal position. In the second experiment, I inactivated the neural activities in the skelomotor areas of the internal globus pallidus (GPi; output area of the basal ganglia) by injecting muscimol while monkey performed overlearned, newly learning and random motor sequence task. It has been reported that dopaminergic neuron in the basal ganglia shows prediction error of reward. This makes many researchers believe that the basal ganglia are a neural substrate for RL by way of temporal difference learning. The inactivation of the GPi resulted in slower reaction times of movements in newly learning condition but almost no changes in overlearned and random conditions. This result shows that the basal ganglia play an important role in motor sequence learning possibly in RL fashion. In conclusion, pre-SMA and SMA area may be ones of brain areas to estimate or to store action values of motor sequences, and the basal ganglia process this information for motor sequence learning in style of RL.