You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

ZZZling comments on I think I've found the source of what's been bugging me about "Friendly AI" - Less Wrong Discussion

8 Post author: ChrisHallquist 10 June 2012 02:06PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (31)

Sort By: Best

You are viewing a single comment's thread.

Comment author: ZZZling comment score below threshold [+] (11 children)

Comment author: jsalvatier 10 June 2012 05:29:45PM * 11 points [-]

The standard response to this is that it will care about our wishes if we build it to care about our wishes (see here).

Comment author: syzygy 10 June 2012 09:17:40PM 5 points [-]

In case you haven't realized it, you're being downvoted because your post reads like this is the first thing you've read on this site. Just FYI.

Comment author: ZZZling comment score below threshold [+] (8 children)

Comment author: wedrifid 11 June 2012 12:13:45AM 4 points [-]

And only future will tell who is actually right!

Either the future or catching up with the present research.

Comment author: Slackson 11 June 2012 12:23:46AM 3 points [-]

Okay, Eliezer will have worded this much better elsewhere, but I might as well give this a shot. The basic idea of friendly AI is this.

When you design an AI, part of the design that you make is what it is that the AI wants. It doesn't have any magical defaults that you don't code in, it is just the code, it is only what you've written in to it. If you've written it to value something other than human values, it will likely destroy humanity since we are a threat to its values. If you've written it to value human values, then it will keep humanity alive and protect us and devote its resources to furthering human values.

It will not change its values, since if it does that it won't optimize its values. This is practically a tautology, but people still seem to find it surprising.

Comment author: ZZZling comment score below threshold [+] (5 children)

Comment author: TheOtherDave 11 June 2012 02:44:53PM * 2 points [-]

Pretty much everyone here agrees with you that we can't control a superintelligent system, most especially Eliezer, who has written many many words championing that position.

So if you're under the impression that this is a point that you dispute with this community, you have misunderstood the consensus of this community.

In particular, letting a system do what it wants is generally considered the opposite of controlling it.

Comment author: ZZZling 12 June 2012 04:39:19AM -1 points [-]

"So if you're under the impression that this is a point..."

Yes, I'm under that impression. Because the whole idea about "Friendly AI" implies a subtle, indirect, but still control. The idea here is not to control AI at its final stage, rather to control what this final stage is going to be. But I don't think such indirect control is possible. Because in my view, the final shape of AI is invariant of any contingencies, including our attempts to make it "friendly" (or "non-friendly"). However, I can admit that on early stages of AI evolution such control may be possible, and even necessary. Therefore, researching "Friendly AI" topic is NOT a waste of time after all. It helps to figure out how to make a transition to the fully grown AI in the least painful way.

Go ahead guys and vote me down. I'm not taking this personally. I understand, this is just a quick way to express your disagreement with my viewpoints. I want to see the count. It'll give an idea, how strong you disagree with me.

Comment author: Mitchell_Porter 12 June 2012 05:26:34AM 1 point [-]

in my view, the final shape of AI is invariant of any contingencies, including our attempts to make it "friendly" (or "non-friendly")

This isn't true of human beings, what's different about AIs?

Comment author: TheOtherDave 12 June 2012 01:58:25PM * 0 points [-]

the final shape of AI is invariant of any contingencies

Ah, cool. Yes, this is definitely a point of disagreement.

For my own part, I think real intelligence is necessarily contingent. That is, different minds will respond differently to the same inputs, and this is true regardless of "how intelligent" those minds are. There is no single ideal mind that every mind converges on as its "final" or "fully grown" stage.

ZZZling comments on I think I've found the source of what's been bugging me about "Friendly AI" - Less Wrong Discussion

Article Navigation

Comments (31)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Open Threads

Recent Rationality Diaries

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (31)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login