You might be thinking of Google's self driving car which seems like it was designed from the ground up with traditional programming. I am thinking of system's like Comma.ai's which use machine learning to train self driving cars, by predicting what a human driver would do.
Of course you can put a regulator on the gas pedal and prevent the AI from speeding. But other issues are more difficult to control. How do you enforce that the Ai should "try to drive with as little risk as possible"? We have very few training examples of accidents, and we can't let the car experiment under real conditions.
My guess on how to solve this issue is to develop a way to "speak" with the AI. So we can see what it is thinking, and tell it what we would prefer it to do. But this is difficult and there is little research on methods to do this, yet.
Google car also uses machine learning. That still doesn't mean that it tries to emulate a human driver. The article doesn't say that the car predicts what a human driver would do.
How do you enforce that the Ai should "try to drive with as little risk as possible"?
There's the example of the Google car waiting for the woman in the wheelchair who chased ducks. That's behavior you get from the way Google algorithm cares about safety that you wouldn't get from emulating human drivers.
If it's worth saying, but not worth its own post, then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one. (Immediately before; refresh the list-of-threads page before posting.)
3. Open Threads should start on Monday, and end on Sunday.
4. Unflag the two options "Notify me of new top level comments on this article" and "