Manfred comments on The Friendly AI Game - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (170)
Make 1 reasonably good cheese cake as judged by a person within a short soft deadline while minimizing the cost to resources made available to it and with out violating property laws as judged by the legal system of the local government within some longer deadline.
To be clear the following do not contribute any additional utility: - Making additional cheese cakes - Making a cheese cake that is better than reasonably good - Making any improvements to the value of the resources available other than making a cheese cake - Anything that happens after the longer deadline.
The short soft deadline is about 1 day. Finishing in 5 hours is slightly better than 6. At 1 day there is a steep inflection. Finishing in more than 25 hours is slightly better than doing nothing.
The long deadline is 1 year.
Committing a property law violation is only slightly worse than doing nothing.
Since we are allowing a technology that does not presently exist I will assume there is another technological change. Specifically there are remotely controllable humanoid robots and people are comfortable interacting with them.
The AGI has available to it: 1 humanoid robot, more money than the cost of ingredients of a cheese cake available at a nearby grocery store, a kitchen, communication access to the person who will judge the reasonableness of the cheese cake and who may potentially provide other resources if requested and who's judgment may be influenced.
As a possible alternative scenario we could assume there are thousands of other similar AGI that each are tasked with making a different pastry. I propose we call them each Marvin.
Let the confectionery conflagration commence!
This seems to fall under the "intelligent rock" category - it's not friendly, only harmless because of the minor nature of its goals.
The minor nature of its goals is the whole point. It is not meant to do what we want because it empathizes with our values and is friendly, but because the thing we actually want it to do really is the best way to accomplish the goals we gave it. Also I would not consider making a cheese cake to be a trivial goal for an AI, there is certainly more to it then the difficult task of distinguishing a spoon from a fork, so this is surely more than just an "intelligent rock".