Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Psilence comments on Understanding your understanding - Less Wrong

69 Post author: SilasBarta 22 March 2010 10:33PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (77)

You are viewing a single comment's thread. Show more comments above.

Comment author: Psilence 23 March 2010 09:14:56AM *  13 points [-]

That's probably closer to the truth than one might think. Once a belief system moves beyond rote memorization of its basic principles and becomes associated with other domains, non-rational beliefs can get very heavily embedded with outside belief networks. The feedback loop that can be created by having just a few anecdotal connections to an already established system would be severe.

The key factor is that, for people who are not strict rationalists already, the "correlation=causation" attitude is quite strong, so any neuronal links I make from new information to outside branches of knowledge can freely flow right back the way they came. Where the rationalist would have to find additional evidence to ingrain a belief, the fundamentalist is free to draw from his outside branches of knowledge to find reverse reinforcement to support the belief he's trying to learn.

Of course, we all do this to a certain extent, bootstrapping our new, tenuous beliefs by looking for associations we can make to older, more familiar territory. But fundamentalists can get through the neuronal rut-treading faster than rationalists, allowing a belief system to become ingrained that much faster.

Also, part of rationalists' training involves maintaining belief system elasticity, so we are ready to shift our conceptions as new information comes along. Fundamentalists, on the other hand, strive in exactly the reverse direction, wanting each neuronal pathway to be as unchanging as possible. There are two main reasons I can think of that this would be important: The one is that God's morality is eternal and unchanging, so the closer we bring our thought patterns out of that messy doubting game, the closer we come to "perfection". The other is that certain idea like adultery or homosexuality are expressly forbidden not to just do, but to even think about. What's a person to do? Well, once you hit the Stage 3 described above, your neural pathways will just naturally flow in the proscribed direction, avoiding extraneous pitfalls that you've edited out.

I remember reading something about this stage with professional chess players a long time ago--a chess master simulates less possible moves in their head than a player with only moderate experience, because past a certain stage, their brain pathways have seen enough games that the obviously "bad" moves simply drop out of their neural net.

Charlie Parker echoed a similar thing about jazz:

“You've got to learn your instrument. Then, you practice, practice, practice. And then, when you finally get up there on the bandstand, forget all that and just wail.”

Unfortunately, the same neural embedding that makes great chess players and musicians possible, also makes cults and other forms of indoctrination possible.

Comment author: reaver121 23 March 2010 10:35:28AM 1 point [-]

Good point. That's why I here argued against thinking about things too long. It's even more important the less rational you are. Before you know it, you are past the point that any evidence can convince you that your opinion is wrong.