V_V comments on Examples of AI's behaving badly - Less Wrong

25 Post author: Stuart_Armstrong 16 July 2015 10:01AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (35)

You are viewing a single comment's thread. Show more comments above.

Comment author: [deleted] 16 July 2015 11:52:45AM 16 points [-]

People do this as well. They wanted to eliminate corruption from public construction projects in a certain country, and created a numbers-based evaluation systems of tenders. The differences in price offered were taken into account with a weight of 1 and the differences in penalties / liquidated damage with a weight of 6. I am not sure what is the best English term for the later, but basically it was the construction company saying if the project is late I am willing to pay X amount of penalty per day. Usually most companies offer something like 0,1% of the price. One company offered 2% which means if they are like 10-15 days late their whole profits are gone, and as this was to be taken into account with a weight of 6, they could offer an outrageous price and the rules still forced the government to accept their offer. It turned out, it was not just a bold gaming of the rules, it was corruption as well: there was no such law that such a penalty offered must also be really enforced in case of late delivery, the government's man can decide to demand less penalty if he feels the vendor is not entirely at fault. So most likely they simply planned to bribe that guy in case if they are late. Thus the new rules simply moved the bribery into a different stage of the process.

When humans are motivated by entirely external incentives like fsck everything let's make as much money on this project as possible, they behave just like the vibrating AI-Messi.

Which means - maybe we need to figure out what the heck is an inner motivation in humans that makes them want to the sensible and how to emulate it.

Comment author: V_V 17 July 2015 03:35:09PM 4 points [-]

People do this as well.

This is known as Goodhart's law or Campbell's law.