Here is a real world control problem: Self driving cars. Companies are currently taking dash cam footage of people driving, and using it to train AIs to drive cars.
There is a serious problem with this. The AIs can learn to predict exactly what a human would do. But humans aren't actually optimal drivers. They make tons of mistakes. They have slow reaction times. They fail to notice things. They don't apply the optimal braking or acceleration, they speed, they don't make optimal turns, etc.
AIs trained on human data end up mimicking all of these imperfections. Then combined with the AIs own imperfections, you get a subpar driver. At best, if the AI is perfect, you get a driver that is equally as good as a human, but not necessarily any better.
Self driving cars are a perfect test case for AI control methods, and a perfect way to encourage mainstream researchers to consider the control problem. There will be many similar cases in the future as AIs start being applied to real world problems in open ended domains. Or wherever there is a hard to define goal function to measure the AI by.
It's not my impression that self driving cars simply try to copy what a human does in any case. The AI don't violate speed limits and generally try to drive with as little risk as possible. Humans drive very differently.
If it's worth saying, but not worth its own post, then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should start on Monday, and end on Sunday.
4. Unflag the two options "Notify me of new top level comments on this article" and "