byrnema comments on Making your explicit reasoning trustworthy - Less Wrong

82 Post author: AnnaSalamon 29 October 2010 12:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (93)

You are viewing a single comment's thread. Show more comments above.

Comment author: byrnema 09 May 2011 03:38:19PM *  8 points [-]

I also agree with MichaelVassar, I think much religious harm comes from using abnormally explicit reasoning.

This is because (I hypothesize that) great moral failures come about when a group of people (often, a religion, but any ideological group) think they've hit upon an absolute "truth" and then expect they can apply this truth to wholly develop an ethical code. The evil comes in when they mistakenly think that morality can be described by some set of universal and self-consistent principles, and they apply a principle valid in one context to another with disastrous results. When they apply the principle to the inappropriate domain, they should feel a twinge of conscience, but they override this twinge with their reason -- they believe in this original principle, and it deduces this thing here, which is correct, so that thing over there that it also deduces must also be correct. In the end, they use reason to override their natural human morality.

The Nazis are the main example I have in mind, but to look at a less painful example, the Catholic church is another example of over-extending principles due to reasoning. Valuing human life and general societal openness to procreation are good values, but insisting that women not use condoms amidst an AIDS epidemic is requiring too much consistency of moral principles.

(Though apparently, I agree even more with user:cousin_it that it is the result of putting ideals of any kind over instinct. It's just that in some cases, the ideal is insisting on consistent, universal moral principles, which religions are fond of doing.)

Comment author: multifoliaterose 09 May 2011 06:44:20PM *  4 points [-]

Thanks for your feedback.

Here I would guess that you're underestimating the influence of (evolutionarily conditioned) straightforwardly base motivations: c.f. the Milgram and Stanford Prison Experiments. I recently ran across this fascinating essay by Ron Jones on his experience running an experiment called "The Third Wave" in his high school class. I would guess that the motivation that he describes (of feeling superior to others) played a significantly larger role than abnormally explicit reasoning in the case of the Nazi regime; that (the appearance of?) abnormally explicit reasoning was a result of this underlying motivation rather than the cause.

There may be an issue generalizing from one example here; what your describing sounds to me closer to why a LW poster might have become a Nazi during Nazi times than why a typical person might have become a Nazi during Nazi times. On the other hand, I find it likely that the originators of the underlying ideas ("Aryan" nationalism, communism, Catholic doctrines) used explicit reasoning more often than the typical person does in coming to their conclusions.

Comment author: byrnema 09 May 2011 07:28:41PM 0 points [-]

I have a question regarding the Milgram experiment. Were the teachers under the impression that the learners were continuing to supply answers voluntarily?

Comment author: Alicorn 09 May 2011 11:20:56PM 2 points [-]

The learner was perceived to initially agree to the experiment, but among the recordings in the programmed resistance was one demanding to be let out.

Comment author: byrnema 10 May 2011 02:13:23AM 2 points [-]

Ah, also this sentence helped my understanding:

Teachers were instructed to treat silence as an incorrect answer and apply the next shock level to the student.

I imagine -- perhaps erroneously -- that I would have tried to obtain the verbal agreement of the learner before continuing. But, for example, this is because I know that continuous subject consent is required whereas this might not have been generally known or true in the early 60s.

Of course, I do see the pattern that this is probably such a case where everyone wants to rate themselves as above average (but they couldn't possibly all be). Still, I will humor my hero-bone by checking out the book and reading about the heroic exceptions, since those must be interesting.

Comment author: multifoliaterose 09 May 2011 10:29:11PM *  0 points [-]

Don't know the answer to your question; now that I look at the Wikipedia page I realize that I should only have referred to the Zimbardo Stanford Prison Experiment (the phenomenon in the Milgram experiment is not what I had in mind).