Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Meetup : AI Safety: Reinforcement Learning with a Corrupted Reward Channel

0 Post author: MathieuRoy 10 July 2017 06:56PM

Discussion article for the meetup : AI Safety: Reinforcement Learning with a Corrupted Reward Channel

WHEN: 10 July 2017 07:00:00PM (-0400)

WHERE: 2920 Ch de la Tour, Montréal, QC

Location: room 3195, Pavillon André-Aisenstadt, Université de Montréal

Contact me, Mati, at 438-886-0994 if you can't find us.

The paper is available here: https://arxiv.org/abs/1705.08417

This is a recent collaboration between DeepMind and Australian National University.

Abstract: << No real-world reward function is perfect. Sensory errors and software bugs may result in RL agents observing higher (or lower) rewards than they should. For example, a reinforcement learning agent may prefer states where a sensory error gives it the maximum reward, but where the true reward is actually small. We formalise this problem as a generalised Markov Decision Problem called Corrupt Reward MDP. Traditional RL methods fare poorly in CRMDPs, even under strong simplifying assumptions and when trying to compensate for the possibly corrupt rewards. Two ways around the problem are investigated. First, by giving the agent richer data, such as in inverse reinforcement learning and semi-supervised reinforcement learning, reward corruption stemming from systematic sensory errors may sometimes be completely managed. Second, by using randomisation to blunt the agent's optimisation, reward corruption can be partially managed under some assumptions. >>

Also see Tom's blog post about this paper here: http://www.tomeveritt.se/paper/2017/05/29/reinforcement-learning-with-corrupted-reward-channel.html

Please do your best to read the paper in advance, so we can have a good discussion about the content.

If necessary, David Krueger will give a run-down of the paper.

Discussion article for the meetup : AI Safety: Reinforcement Learning with a Corrupted Reward Channel

Comments (2)

Comment author: waveman 11 July 2017 01:35:59AM 1 point [-]

Can I suggest you put the name of the city at the start of the title, so that everyone doesn't need to go into the posting to check is it their city?

Others do this. Thanks,

Comment author: MathieuRoy 27 July 2017 07:53:21AM 0 points [-]

Just saw your comment. Thanks for letting me know.