Dopamine and Reward Prediction Error

I am back from the IBAGS conference and full of new information! I plan to blog about tons of amazing things over the next month or so, but today we'll start with some foundation building.

Dopamine nails (source)

The IBAGS (international basal ganglia society) meeting is all about the basal ganglia (which includes the striatum), and as you may know, dopamine is a super important molecule for the proper function of the striatum (it is the dopamine cells that die in Parkinson's Disease).

There were many fantastic talks during the IBAGS meeting and almost a third of them showed the exact same figure on one of their slides. So much so that everyone would start to laugh when someone showed it. And as you may have guessed, it is about dopamine. Here it is:

Schultz 1998 Figure 2

This figure is the basis for the belief that dopamine represents a 'reward prediction error'.  Let me explain. The scattered dots on the lower half of each panel represent action potentials from individual dopaminergic neurons. The x axis it time in seconds. The black columns above them are a histogram showing how much firing is going on at each point in time. When the black columns are tall, there was more dopamine neuron firing. You can see that the height of the black columns matches up with the density of the scattered dots below them.

During 'reward learning', an animal is trained to associate a stimulus (like a tone or a flash of light) with getting a reward (like a drop of water or juice). These three panels show how dopamine responds to this whole process. The first panel shows that when there is no stimulus (CS) and the reward is a surprise, the dopamine neurons respond very strongly to it. The second panel shows that when there is a stimulus that tells the animal that a reward is soon to come, the dopamine neurons respond to the stimulus, but not to the reward. Finally the third panel shows that when there is a stimulus the dopamine neurons respond to it, but if the reward (R) never comes, the dopamine neurons actually stop firing when the reward should have happened.

What is so fascinating about this is that it shows dopamine neurons do not just fire in response to reward, they encode the actual reward with respect to the expected reward. In the author's words:

"Dopamine neurons report rewards relative to their prediction rather than signaling primary rewards unconditionally (Fig. 2). The dopamine response is positive (activation) when primary rewards occur without being predicted. The response is nil when rewards occur as predicted. The response is negative (depression) when predicted rewards are omitted. Thus dopamine neurons report primary rewards according to the difference between the occurrence and the prediction of reward, which can be termed an error in the prediction of reward..." Schultz 1998

This finding is so important to researchers now because it shows that dopamine neurons can encode learning rules. Dopamine neurons constantly and dynamically tell the rest of the brain which stimuli lead to reward, and which stimuli don't. The implications here for pathological learning are huge as well. Mis-signalling in dopamine neurons could lead to an inability to tell what is rewarding and what is not.

© TheCellularScale



Schultz W (1998). Predictive reward signal of dopamine neurons. Journal of neurophysiology, 80 (1), 1-27 PMID: 9658025

Honoring a Legend

The Cellular Scale is at the International Basal Ganglia Society meeting this week (#IBAGS2013), and finally has internet!

Sunrise over the Gulf of Aqaba (I took this picture)

It's already been two days of conferencing, and I plan to mainly write some follow up posts when I get back. But I will just briefly mention the "Lifetime Member" lecture that was given on the first evening of the conference.

Mahlon Delong (source)

This year's lifetime member is Mahlon DeLong.
I've written before about deep brain stimulation (DBS) as a treatment for Parkinson's Disease, and DeLong has done some fascinating work that has lead up to DBS in the  subthalamic nucleus (STN).

One particular treat was to see a video during the talk of the very first attempt at alleviating Parkinson's symptoms through a subthalamotomy, the lesion of the subthalamic nucleus.

A Parkinson's Disease monkey was given the subthalamotomy on only one side of the brain and the video shows Mahlon DeLong interacting with the monkey and noting that it's treated side is less stiff than the untreated side. A second video shows the monkey later able to move its arm with no problems.

It was exciting to see this sort of 'moment of discovery' from 1989. There were no cries of "Eureka!" or anything it was more of a 'hm, interesting' tone. You actually hear his post-doc on the video saying
(paraphrasing from memory) "the right side has better tone, at least Mahlon thinks so" and then start laughing.

(source)

One other cool thing about Dr. DeLong is that he is Muhammad Ali's physician.

© TheCellularScale