gwern comments on People who "don't rationalize"? [Help Rationality Group figure it out] - Less Wrong

12 Post author: Mercurial 02 March 2012 11:38PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (85)

You are viewing a single comment's thread. Show more comments above.

Comment author: Mercurial 03 March 2012 06:54:46AM 7 points [-]

I will say that it doesn't even seem to be possible for there to be people who don't rationalize. (Or enough that you're at all likely to find them.)

You'd think not. Yet even Eliezer seems to think that one of our case studies really, truly might not ever rationalize and possibly never has before. This seems to be a case of a beautiful, sane theory beaten to death by a small gang of brutal facts.

"Some", "signs", "rather". These words all show signs of being rather belief in belief. I notice you don't say, "Some of these people are high-level rationalists," just that they show warning signs of being so. What does this really mean?

It means that I don't know how to measure how strong someone's rationality skills are other than talking to others whom I intuitively want to say are good rationalists and comparing notes. So I'm hedging my assertions. But to whatever degree several people at the Singularity Institute are able to figure out who is or is not a reasonably good rationalist, some of our sample "non-rationalizers" appear to us to be good rationalists, and some appear not to be so.

Also, could you explain what you mean by "seem to have little clue what Tarski is for"?

Sure. We tell them the kinds of situations in which Tarski is useful, including some personal examples of our own applications of it, and they just blink at us and completely fail to relate. For instance, I might say, "So once I was walking past a pizza place and smelled pizza. Cheese turns out to be really bad for me, but at the time I was hungry. So I watched my mind construct arguments like, 'I haven't gotten much calcium for the last while.'" Nothing of this sort - fake justification, selective search, nothing - seems to connect to something they can relate to. So they just don't see where they'd ever use Tarski.

And yes, we've had at least one person be openly skeptical that anyone could possibly find Tarski useful because he didn't think anyone rationalized the way we were describing. And another of our case studies seemed to know rationalization only as a joke. ("The cake has fewer calories and doesn't count if I eat it while standing, right?")

Comment author: gwern 03 March 2012 08:28:51AM 15 points [-]

Have you actually tested them for rationalizing? My own beliefs are that it's more likely to run into someone who rationalizes so much they are blind to their own rationalizing (and so can't recall any) than someone who is inhumanly honest.

(Tests in this case would include checking for hindsight bias, which is classic rationalizing, and having them do that test on YourMorals whose name I forget where you're given two summaries of studies for and against gun control and asked to criticize them - usually showing imbalance towards your favored side. But you're a LWer, I'm sure you can think of other tests.)