Comment on Stupid Questions Open Thread

Fri, 30 Dec 2011 02:11:59 GMT

As I understand the terminology, AI that only respects some humans' preferences is uFAI by definition. Thus:

a friendly AI that is created in some researcher's personal image

is actually unFriendly, as Eliezer uses the term. Thus, the researcher you describe is already an "uFAI researcher"

It also removes the problem of different morals/values. Some people believe that we should look at total utility, other people believe we should consider only average utility. Some people believe abstract values matter, some people believe consequences of actions matter most. Here too the solution of an AI that looks at a representative set of all human values is the solution that all people can agree on as most "fair".

What do you mean by "representative set of all human values"? Is there any reason to that the resulting moral theory would be acceptable to implement on everyone?

Discuss