Peter_de_Blanc comments on The Sword of Good - Less Wrong

85 Post author: Eliezer_Yudkowsky 03 September 2009 12:53AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (292)

You are viewing a single comment's thread. Show more comments above.

Comment author: Peter_de_Blanc 03 September 2009 04:33:22PM *  14 points [-]

his own moral code, which assumes power corrupts

Hold on. How can a moral code say anything about questions of fact, such as whether or not power corrupts?

Comment author: SilasBarta 03 September 2009 05:26:15PM 6 points [-]

Because "corrupt" is a morally-loaded term.

Comment author: Peter_de_Blanc 03 September 2009 05:38:30PM 14 points [-]

It seems to me that "power corrupts" means "power changes goal content," and that's a purely factual claim.

Comment author: SilasBarta 03 September 2009 05:45:06PM *  9 points [-]

It doesn't mean that. It means something more like "power changes the empowered's utility function in a way others deem immoral". (ETA simplified)

ETA: Just to make the point clearer, there are many things that change an individual's goal content but are not considered corrupting. For example, trying new foods will generally make you divert more effort to finding one kind of food (that you didn't know you liked). Having children of your own makes you more favorable to children in general. But we don't say, and people generally don't believe, "having children corrupts" or "trying new foods corrupts".

Comment author: Eliezer_Yudkowsky 03 September 2009 07:16:33PM 6 points [-]

Okay, but that's still a factual claim underneath the moral one.

It's a bit of argumentum ad webcomicum, but http://www.agirlandherfed.com/comic/?375 is not something I find particularly implausible. There was Marcus Aurelius.

Comment author: PlatypusNinja 04 September 2009 01:32:23AM 7 points [-]

Also: it seems like a really poor plan, in the long term, for the fate of the entire plane to rest on the sanity of one dude. If Hirou kept the sword, he could maybe try to work with the wizards -- ask them to spend one day per week healing people, make sure the crops do okay, etc. Things maybe wouldn't be perfect, but at least he wouldn't be running the risk of everybody-dies.

Comment author: kpreid 01 June 2012 02:15:00PM 6 points [-]

Link's broken. Is this guess the page in question?

Comment author: Eliezer_Yudkowsky 01 June 2012 04:24:03PM 2 points [-]

Yup!

Comment author: thomblake 04 September 2009 01:12:12PM *  2 points [-]

And then there are those of us who take moral claims to be factual claims.

Comment author: SilasBarta 03 September 2009 07:38:55PM 2 points [-]

Okay, but in any case, regarding the issue at hand, "power corrupts" is not a purely factual claim. (And I thought that hybrid claims get counted as moral by default, since that's the most useful for discussion, but I could be wrong.)

Comment author: Peter_de_Blanc 03 September 2009 09:03:02PM 1 point [-]

Then you need to separate the factual claim and the moral claim, and discuss them separately. The factual claim would be, "power changes goal content in this particular way", and the moral claim is, "...and this is bad."

Comment author: SilasBarta 06 September 2009 10:03:10PM *  0 points [-]

Is this fair though? Let's say the passage had been, "... his position that it is immoral to possess nuclear weapons". That too breaks down into a factual and moral claim.

Moral: "it is wrong to possess a weapon with massive, unfocused destructive power"

Factual: "The devices we currently call nuclear weapons inflict massive, unfocused destruction."

Would you object to "his position that it is immoral to posses nuclear weapons" on the grounds that "you need to separate the factual and moral claims"?

Comment author: Eliezer_Yudkowsky 06 September 2009 10:42:43PM 6 points [-]

Well, in fact it would be highly helpful to separate the claims here, even though the factual part is uncontroversial, because it makes it clear what argument is being made, exactly.

And in this case it's uncertain/controversial how much power actually changes behavior, who it changes, how reliably; and this is the key issue, whereas the moral concept that "the behavior of killing everyone who disagrees with you, is wrong" is relatively uncontroversial among us. So calling this a moral claim when the key disputed part is actually a factual claim is a bad idea.

Comment author: Wei_Dai 03 September 2009 07:41:24PM 3 points [-]

What's the evolutionary explanation for power not corrupting?

Comment author: MichaelVassar 05 September 2009 04:28:18AM 5 points [-]

Evolution doesn't do most things. Doing things requires oceans of blood for every little adaptation and humans haven't had power for all that long.
Toddlers need to learn how to hide. How's that for failing to evolve knowledge of the obvious (to a human brain) and absurdly useful.

Comment author: SilasBarta 03 September 2009 07:53:33PM 3 points [-]

Be careful you don't end up explaining two contradictory outcomes equally well, thus proving you have zero knowledge on evolution's effect on power and corruption!

Comment author: PlatypusNinja 04 September 2009 01:26:45AM 0 points [-]

I think my concern about "power corrupts" is this: humans have a strong drive to improve things. We need projects, we need challenges. When this guy gets unlimited power, he's going to take two or three passes over everything and make sure everybody's happy, and then I'm worried he's going to get very, very bored. With an infinite lifespan and unlimited power, it's sort of inevitable.

What do you do, when you're omnipotent and undying, and you realize you're going mad with boredom?

Does "unlimited power" include the power to make yourself not bored?