lukeprog comments on A Crash Course in the Neuroscience of Human Motivation - Less Wrong

119 Post author: lukeprog 19 August 2011 09:15PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (90)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 20 August 2011 01:52:33PM *  3 points [-]

The footnote in that sentence leads you to this short paper linked from my bibliography, which reads:

The efficiency and accuracy with which neural activity can code the value of a stimulus (such as liquid volume) can be greatly increased if neurons make use of information about the probabilities of potential reward values. Neural activity can then be devoted to representing probable values at the expense of improbable values. Our evidence suggests that the transient dopamine response to conditioned stimuli may carry information on expected reward value, and previous work shows that the more sustained activity of dopamine neurons reflects a measure of reward uncertainty such as variance. If the system possesses prior information consisting of the expected value and variance of reward, then this information need not be represented redundantly at the time of reward. Discarding this old information may be achieved by subtracting the expected value from the absolute reward value and then dividing by the variance. Analogous normalization processes appear to occur in early visual neurons. It is not known to what extent the normalization processes observed in dopamine neurons are actually performed in dopamine neurons as opposed to their afferent input structures. Because the new information is by definition precisely the information that the system needs to learn, the activity of dopamine neurons would be an appropriate teaching signal.

I'm sorry I don't have more time to go into this, hopefully the paper link helps.

Comment author: TobyBartels 21 August 2011 08:00:06PM 12 points [-]

Thanks, I don't know why I didn't follow the footnote.

But if I had, I would have added to my comment that the cited paper confirms my expectations. The function that they describe (as in your quoted paragraph) does preserve ordering, and seems to have nothing to do with the compressive functions described at Wikipedia. (The paper also doesn't use that term; the case-independent string ‘compr’ doesn't appear in it at all.)

But actually, the point of the article seems to be that the function from reward magnitude to dopamine rate varies with time, being renormalised from time to time to be most sensitive (literally, having highest derivative) at the most likely inputs, which I did not get from your post at all. But if I were an editor, wanting your post to best reflect the article without getting any longer, I'd suggest just changing ‘compressive function’ to ‘variable function’ and removing the irrelevant link.

Not that any of this should detract from your otherwise excellent and everywhere interest post!

Comment author: lukeprog 22 August 2011 02:44:34AM 7 points [-]

Gack! I'm just completely wrong about this one. Thanks for reading so closely and correcting my mistake!

Comment author: lessdazed 22 August 2011 03:46:21AM 1 point [-]

Upvoted.

Comment author: TobyBartels 22 August 2011 07:42:07PM 1 point [-]

You're welcome then!

Comment author: lessdazed 22 August 2011 03:49:57AM 1 point [-]

Upvoted for reading closely and then reading a paper in the footnotes.