= 27d567bc2d48d9f40e9ccc20ea370b15)
Vladimir_Nesov comments on The Sword of Good - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (292)
Nope, they didn't get that part wrong.
Look, you should know me well enough by now to know that I don't keep my stories on nice safe moral territory.
A happy ending here is not guaranteed. But think about this very carefully. Are you sure you'd have turned the Sword on Vhazhar? They don't have the same options we do.
I assume that the sword tests the correspondence of person's intentions (plan) to their preference. If the sword uses a static concept of preference that comes with the sword instead, why would Vhazhar be interested in sword's standard of preference? Thus, given that the Vhazhar's plan involves control over the fabric of the World, the plan must be sound and result in correct installation of Vhazhar's preference in the rules of the world. This excludes the technical worries about the failure modes of human mind in wielding too much power (which is how I initially interpreted "personal control" -- as a recipe for failure modes).
I'm not sure what it means for the other people's preferences (and specifically mine). I can't exclude the possibility that it's worse than the do-nothing option, but it doesn't seem obviously so either, given psychological unity of humans. From what I know, on the spot I'd favor Vhazhar's personal preference, if the better alternative is unlikely, given that this choice instantly wards off existential risk and lack of progress.
No, it's the Sword of GOOD. It tests whether you're GOOD, not any of this other stuff.
It should be obvious that the sword doesn't test how well your plans correspond to what you think you want! Otherwise Hirou would have been vaporized.
Wasn't it established that this world's conception of "good" and "evil" are messed up? Why should he trust that the sword really works exactly as advertised?
Only assuming that the sword is impulsive. If you take into account Hirou's overall role in the events, this role could be judged good, if only by the final decision.
If the sword judges not plans, but preference, then failing 9 out of 10 people means that it's pretty selective among humans and probably people it selects and their values aren't representative (act in the interests) of the humanity as whole.
If the Sword of Good tested whether you're good, Hirou would have been vapourized, because he was obviously not good. He was at the very least an accomplice to murderers, a racist, and a killer. The Sword of Good may not have vapourized Charles Manson, Richard Nixon, Hitler, or most suicide bombers, either. The Sword of Good tests whether you think you are good, not whether your actions are good.
Strangely, the sword kills nine out of ten people who try to wield it. However, if you knew the sword could only be wielded by a good person, you'd only try to pick it up if you thought you were good, which happens to be the criteria you must fulfil in order to pick up the sword. Essentially, if you think you can wield the Sword of Good, you can.
Well, he was clearly redeemable, at least. It didn't take very much for him to let go of his assumptions, just a few words from someone he thought was an enemy. Making dumb mistakes, even ones with dire consequences, doesn't necessarily make you not Good.
What, realistically, does it mean to be irredeemable? Was Dolf irredeemable? Selena? Is the difference between them and Hirou simply the fact that Hirou realized he was doing bad, and they didn't? Why should that be sufficient to redeem him? Mistakes are not accidents; mistakenly killing someone is still murder.
Surely if awareness and repentance of the immoral nature of your actions makes you Good, the reverse - lack of awareness - means animals that kills other animals without regret are more evil than people who kill other people and regret it.
No, it's manslaughter.
If you believe someone is evil, hunt them down and kill them, and afterward realize they weren't, it was a mistake. It was also murder. It's not as though you killed in self defense or accidentally dropped an air conditioner on them. Manslaughter is not a defense that can be employed simply because you changed your mind.
Perhaps I should clarify: I don't mean "mistake" in that "he mistook his wife for a burglar and killed her". That's manslaughter. I mean "mistake" in that "he mistakenly murdered a good person instead of a bad one". Ba gur bgure unaq, jura Uvebh xvyyrq Qbys ng gur raq, ur jnfa'g znxvat n zvfgnxr (ubjrire, V fgvyy guvax vg jnf zheqre).
You present a compelling argument that murder can be a morally blameless---even praiseworthy---act. I do not believe this was your intention.
To be clear, you believe that, right wedrifid? I came this close to downvoting before I deduced the context.
Suppose you're a police officer trying to arrest someone for a crime, and there is ample evidence that the person you are trying to arrest is indeed guilty of that crime. The person resists arrest, and you end up killing the person instead of making a successful capture. Are you a murderer?
Does it matter if it turns out that the evidence against this person turns out to have been forged (by someone else)?
If you have no intention of killing them and they die as a side effect of your actions, it's an accident, and manslaughter. If you kill them because you realize you can't arrest them, it's murder, complete with intention of malice. However, the fact that your actions are sanctioned by the state is obviously not a defense (a la Nuremberg), and so there's no point in adding "police officer" to the example.
You could ask if I thought executing someone who was framed would be considered murder, but since I view all manner of execution murder, guilty or no, there's no use.
I perceive that you have not yet learned to use the logic of the Phoenix.
Care to elaborate on that rather cryptic remark?
logic of the phoenix?
Doing a bad thing does not necessarily make one a bad person. Though it helps.
You are using two definitions of "good" - how much good your actions cause, and how good you believe yourself to be. Neither of those is used by the sword; rather, some sort of virtue-ethics definition - I suspect motive.
So a sincerely evil person would pass with flying colors?
I assumed the sword tested compliance with the current CEV of the human race.
Why just the human race? Orcs are people too (at least in this story).
Good catch. Yes, of course.
Presumably, actual mutants are unlikely, with most "evil" people actually just holding mistaken (about their actual preference) moral beliefs. If the sword is an external moral authority, it's harder to see why one would consult it.
On the other hand, sword checks soundness of the plan against some preference, which is an important step that is absent if one doesn't consult the sword, which can justify accepting a somewhat mismatched preference if that allows to use the test.
This passes the choice of mismatching preferences to a different situation. If the sword tests person's preference, then protagonist's choice is between lack of progress or unlikely good outcome and (if Vhazhar's plan is sound) verified installation of Vhazhar's preference, with the latter presumably close to others' preference, thus being a moderately good option. If the sword tests some kind of standard preference, this standard preference is presumably also close to Vhazhar's preference, thus Vhazhar faces a choice between trying to install his own preference through unverified process, which can go through all kinds of failure modes, and using the sword to test the reliability of his plan.
The fact that Vhazhar is willing to use the sword to test the soundness of his plan, when the failed test means his death, shows that he prefers leaving the rest of the world be to incorrectly changing it. This is a strong signal that should've been part of the information given to protagonist for making the decision.