AdeleneDawner comments on Welcome to Heaven - Less Wrong

23 Post author: denisbider 25 January 2010 11:22PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (242)

Comments (242)

Sort By: Best

Show:

You are viewing a single comment's thread. Show more comments above.

Comment author: AdeleneDawner 26 January 2010 01:44:32PM 1 point [-]

If the FAI values that we value independence, and values that we value autonomy - which I think it would have to, to be considered properly Friendly - and if wireheading is an threat to our ability to maintain those values, it doesn't make sense that the FAI would make wireheading available for the asking. It makes much more sense that the FAI would actively protect us from wireheading as it would from any other existential threat, in that case.

(Also, just because it would protect us from existential threats, that wouldn't imply that it would protect us from non-existential ones. Part of the idea is that it's very smart: It can figure out the balance of protecting and not-protecting that best preserves its values, and by extension ours.)

Comment author: denisbider comment score below threshold [+] (82 children)

Comment author: AdeleneDawner 26 January 2010 02:09:39PM * 2 points [-]

Those aren't existential threats, especially if we had enough medical technology to easily deal with any negative side effects of them.

ETA: When I posted this, the above comment was only "Just like the government tries to "protect" you from marijuana, alcohol, cigarettes and fatty food, you mean?".

Comment author: denisbider 26 January 2010 02:17:53PM 0 points [-]

The analogy is between governmental paternalism today, with the technology today, and FAI paternalism tomorrow, with the technology tomorrow.

The analogy is not between FAI tomorrow, and governmental paternalism today, but with the technology of tomorrow. That would be a conflicted state of affairs.

Today, smoking is an existential threat, and governmental paternalism is... paternalism.

Comment author: AdeleneDawner 26 January 2010 02:33:51PM 2 points [-]

The things you mentioned are not existential threats; they do not threaten the existence of the human species. They may reduce the lifespan, or total experienced hedonic pleasure, of people who indulge in them, but as I pointed out in my parenthetical, that's a separate issue.

FAI paternalism is an almost entirely different issue than governmental paternalism, because a FAI an a government are two very different kinds of things. A properly friendly AI can be trusted not to be corrupt, for example, or to be swayed by special interest groups, or to be confused by bad science or religious arguments.

Comment author: denisbider comment score below threshold [+] (78 children)

Comment author: Alicorn 26 January 2010 03:00:02PM 4 points [-]

Merely having the inability to regret an occurrence doesn't make the occurrence coincide with one's preferences. I couldn't regret an unexpected, instantaneous death from which I was never revived, either; I emphatically don't prefer one.

Comment author: denisbider 26 January 2010 03:06:48PM * 0 points [-]

But wire-heading is not death. It is the opposite - the most fulfilling experience possible, to which everything else pales in comparison.

It seems you think paternalism is okay if it is pure in intent and flawless in execution.

It has been shown that vulnerability to smoking addiction is due to a certain gene. Suppose we could create a virus that would silently spread through the human population and fix this gene in everyone, willing or not. Suppose our intent is pure, and we know that this virus would operate flawlessly, only affecting this gene and having no other effects.

Would you be in favor of releasing this virus?

Comment author: RobinZ 26 January 2010 03:46:21PM 1 point [-]

But wire-heading is not death. It is the opposite - the most fulfilling experience possible, to which everything else pales in comparison.

..."fulfilling"? Wire-heading only fulfills "make me happy" - it doesn't fulfill any other goal that a person may have.

"Fulfilling" - in the sense of "To accomplish or carry into effect, as an intention, promise, or prophecy, a desire, prayer, or requirement, etc.; to complete by performance; to answer the requisitions of; to bring to pass, as a purpose or design; to effectuate" (Webster 1913) - is precisely what wire-heading cannot do.

Comment author: denisbider 26 January 2010 03:51:04PM * -2 points [-]

Your other goals are immaterial and pointless to the outside world.

Nevertheless, suppose the FAI respects such a desire. This is questionable, because in the FAI's mind, this is tantamount to letting a depressed patient stay depressed, simply because a neurotransmitter imbalance causes them to want to stay depressed. But suppose it respects this tendency.

In that case, the cheapest way to satisfy your desire, in terms of consumption of resources, is to create a simulation where you feel like you are thinking, learning and exploring, though in reality your brain is in a vat.

You'd probably be better off just being happy and sharing in the FAI's infinite wisdom.

continue this thread »

Comment author: Alicorn 26 January 2010 03:10:45PM 0 points [-]

It seems you think paternalism is okay if it is pure in intent and flawless in execution.

Whoa whoa whoa wait what? No. Not under a blanket description like that, at any rate. If you want to wirehead, and that's your considered and stable desire, I say go for it. Have a blast. Just don't drag us into it.

Would you be in favor of releasing this virus?

No. I'd be in favor of making it available in a controlled non-contagious form to individuals who were interested, though.

Comment author: denisbider 26 January 2010 03:14:48PM 0 points [-]

Apologies, Alicorn - I was confusing you with Adelene. I was paying all attention to the content and not enough to who is the author.

Only the first paragraph (but wire-heading is not death) is directed at your comment. The rest is actually directed at Adelene.

continue this thread »

Comment author: AdeleneDawner 26 January 2010 03:01:16PM 3 points [-]

I maintain that an AI that would do that isn't Friendly.

I believe that my definition of Friendliness is in keeping with the standard definition that's in use here.

How are you defining Friendliness, that you would consider an AI that would wirehead someone against their will to be Friendly?

Comment author: denisbider 26 January 2010 03:09:27PM 1 point [-]

Is it friendly to rescue a drunk friend who is about to commit suicide, knowing that they'll come to their senses? Or is it friendly to let them die, because their current preference is to die?

Comment author: AdeleneDawner 26 January 2010 03:13:01PM 4 points [-]

That depends on whether they decided to commit suicide while in a normal-for-them frame of mind, not on their current preference. The first part of the question implies that they didn't, in which case the correct response is to rescue them, wait for them to get sober, and talk it out - and then they can commit suicide, if they still feel the need.

Comment author: denisbider 26 January 2010 03:20:02PM 1 point [-]

Very well, then. Next example. Your friend is depressed, and they want to commit suicide. You know that their real problem is a neurotransmitter imbalance that can be easily fixed. However, that same neurotransmitter imbalance is depriving them of any will to fix it, and in fact they refuse to cooperate. You know that if you fix their imbalance regardless, they will be happy, they will live a fulfilled life, and they will be grateful to you for it. Is it friendly to intervene and fix the imbalance, or is it friendly to let them die, seeing as depression and thoughts of suicide are a normal-for-them frame of mind?

continue this thread »

Comment author: aausch 27 January 2010 04:21:27PM 0 points [-]

Probably the "friendly" action would be to create an un-drunk copy of them, and ask the copy to decide.

Comment author: RobinZ 27 January 2010 05:46:48PM 0 points [-]

And what do you do with the copy? Kill it?

continue this thread »

AdeleneDawner comments on Welcome to Heaven - Less Wrong

Article Navigation

Comments (242)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (242)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login