You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

TimS comments on Stupid Questions Open Thread - Less Wrong Discussion

42 Post author: Costanza 29 December 2011 11:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (265)

Comments (265)

You are viewing a single comment's thread. Show more comments above.

Comment author: TimS 30 December 2011 02:11:59AM *  1 point [-]

As I understand the terminology, AI that only respects some humans' preferences is uFAI by definition. Thus:

a friendly AI that is created in some researcher's personal image

is actually unFriendly, as Eliezer uses the term. Thus, the researcher you describe is already an "uFAI researcher"

It also removes the problem of different morals/values. Some people believe that we should look at total utility, other people believe we should consider only average utility. Some people believe abstract values matter, some people believe consequences of actions matter most. Here too the solution of an AI that looks at a representative set of all human values is the solution that all people can agree on as most "fair".

What do you mean by "representative set of all human values"? Is there any reason to that the resulting moral theory would be acceptable to implement on everyone?

Comment author: Zed 30 December 2011 02:22:36AM *  1 point [-]

[a "friendly" AI] is actually unFriendly, as Eliezer uses the term

Absolutely. I used "friendly" AI (with scare quotes) to denote it's not really FAI, but I don't know if there's a better term for it. It's not the same as uFAI because Eliezer's personal utopia is not likely to be valueless by my standards, whereas a generic uFAI is terrible from any human point of view (paperclip universe, etc).

Comment author: TimS 30 December 2011 02:40:31AM -1 points [-]

I guess it just doesn't bother me that uFAI includes both indifferent AI and malicious AI. I honestly think that indifferent AI is much more likely than malicious (Clippy is malicious, but awfully unlikely), but that's not good for humanity's future either.

Powered by Reddit Powered by Reddit