Val comments on Examples of AI's behaving badly - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (35)
Another, famous, example. At one time, somewhere in India there were a lot of cobras, which are dangerous. So the government (it happened to be the British Raj at the time) decided to offer a bounty for dead cobras. That worked for a while, until people figured out that they could breed cobras for the bounty. Then the government worked out what was going on, and cancelled the bounty. So then the cobra breeders released all their now-valueless cobras into the wild.
(According to Wikipedia this particular instance isn't actually well documented, but a similar one involving rats in Hanoi is.)
This effect also exists in software development:
http://thedailywtf.com/articles/The-Defect-Black-Market
Famous Dilbert cartoon on this topic.