Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

pjeby comments on That Magical Click - Less Wrong

54 Post author: Eliezer_Yudkowsky 20 January 2010 04:35PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (393)

You are viewing a single comment's thread.

Comment author: pjeby 20 January 2010 06:34:15PM 130 points [-]

My best guess is that clickiness has something to do with failure to compartmentalize - missing, or failing to use, the mental gear that lets human beings believe two contradictory things at the same time. Clicky people would tend to be people who take all of their beliefs at face value.

One of the things that I've noticed about this is that most people do not expect to understand things. For most people, the universe is a mysterious place filled with random events beyond their ability to comprehend or control. Think "guessing the teacher's password", but not just in school or knowledge, but about everything.

Such people have no problem with the idea of magic, because everything is magic to them, even science.

An anecdote: once, when I still worked as software developer/department manager in a corporation, my boss was congratulating me on a million dollar project (revenue, not cost) that my team had just turned in precisely on time with no crises.

Well, not congratulating me, exactly. He was saying, "wow, that turned out really well", and I felt oddly uncomfortable. After getting off the phone, I realized a day or so later that he was talking about it like it was luck, like, "wow, what nice weather we had."

So I called him back and had a little chat about it. The idea that the project had succeeded because I designed it that way had not occurred to him, and the idea that I had done it by the way I negotiated the requirements in the first place -- as opposed to heroic efforts during the project -- was quite an eye opener for him.

Fortunately, he (and his boss) were "clicky" enough in other areas (i.e., they didn't believe computers were magic, for example) that I was able to make the math of what I was doing click for them at that "teachable moment".

Unfortunately, most people, in most areas of their lives treat everything as magic. They're not used to being able to understand or control anything but the simplest of things, so it doesn't occur to them to even try. Instead, they just go along with whatever everybody else is thinking or doing.

For such (most) people, reality is social, rather than something you understand/ control.

(Side note: I find myself often trying to find a way to express grasp/control as a pair, because really the two are the same. If you really grasp something, you should be able to control it, at least in principle.)

Comment author: arundelo 20 January 2010 10:49:36PM *  30 points [-]

Such people have no problem with the idea of magic, because everything is magic to them, even science.

Years ago I and three other people were training for a tech support job. Our trainer was explaining something (the tracert command) but I didn't understand it because his explanation didn't seem to make sense. After asking him more questions about it, I realized from his contradictory answers that he didn't understand it either. The reason I mention this is that my three fellow trainees had no problem with his explanation, one even explicitly saying that she thought it made perfect sense.

Comment author: Eliezer_Yudkowsky 21 January 2010 12:24:24AM 32 points [-]

Huh. I guess that if I tell myself, "Most people simply do not expect reality to make sense, and are trying to do entirely different things when they engage in the social activity of talking about it", then I do feel a little less confused.

Comment author: pjeby 21 January 2010 02:15:03AM 31 points [-]

Most people simply do not expect reality to make sense

More precisely, different people are probably using different definitions of "make sense"... and you might find it easier to make sense of if you had a more detailed understanding of the ways in which people "make sense". (Certainly, it's what helped me become aware of the issue in the first place.)

So, here are some short snippets from the book "Using Your Brain For A Change", wherein the author comments on various cognitive strategies he's observed people using in order to decide whether they "understand" something:

There are several kinds of understanding, and some of them are a lot more useful than others. One kind of understanding allows you to justify things, and gives you reasons for not being able to do anything different....

A second kind of understanding simply allows you to have a good feeling: "Ahhhh." It's sort of like salivating to a bell: it's a conditioned response, and all you get is that good feeling. That's the kind of thing that can lead to saying, "Oh, yes, 'ego' is that one up there on the chart. I've seen that before; yes, I understand." That kind of understanding also doesn't teach you to be able to do anything.

A third kind of understanding allows you to talk about things with important sounding concepts, and sometimes even equations.... Concepts can be useful, but only if they have an experiential basis [i.e. "near" beliefs that "pay rent"], and only if they allow you to do something different.

Obviously, we are talking mostly about "clicking" being something more like this latter category of sense-making, but the author actually did mention how certain kinds of "fuzzy" understanding would actually be more helpful in social interaction:

However, a fuzzy, bright understanding will be good for some things. For example, this is probably someone who would be lots of fun at a party. She'll be a very responsive person, because all she needs to do to feel like she understands what someone says is to fuzz up her [mental] pictures. It doesn't take a lot of information to be able to make a bright, fuzzy movie. She can do that really quickly, and then have a lot of feelings watching that bright movie. Her kind of understanding is the kind I talked about earlier, that doesn't have much to do with the outside world. It helps her feel better, but it won't be much help in coping with actual problems.

Most of the chapter concerned itself with various cognitive strategies of detailed understanding used by a scientist, a pilot, an engineer, and so on, but it also pointed out:

What I want you all to realize is that all of you are in the same position as that ... woman who fuzzes images. No matter how good you think your process of understanding is, there will always be times and places where another process would work much better for you. Earlier someone gave us the process a scientist used -- economical little pictures with diagrams. That will work marvelously well for [understanding] the physical world, but I'll predict that person has difficulties understanding people -- a common problem for scientists. (Man: Yes, that's true.)

Anyway, that chapter was a big clue for me towards "clicking" on the idea that the first two obstacles to be overcome in communicating a new concept are 1) getting people to realize that there's something to "get", and 2) getting them to get that they don't already "get" it. (And both of these can be quite difficult, especially if the other person thinks they have a higher social status than you.)

Comment author: MichaelGR 21 January 2010 03:32:55PM 6 points [-]

Would you recommend that book? ("Using Your Brain For A Change")

Is the rest of it insightful too, or did you quote the only good part?

Comment author: pjeby 22 January 2010 02:33:50AM 12 points [-]

Is the rest of it insightful too, or did you quote the only good part?

There are a lot of other good parts, especially if you care more about practice than theory. However, I find that personally, I can't make use of many of the techniques provided without the assistance of a partner to co-ordinate the exercises. It's too difficult to pay attention to both the steps in the book and what's going on in my head at the same time.

Comment author: sketerpot 21 January 2010 02:28:12AM *  23 points [-]

I'm still confused, but now my eyes are wide with horror, too. I don't dispute what pjeby said; in retrospect it seems terribly obvious. But how can we deal with it? Is there any way to get someone to start expecting reality to make sense?

I have a TA job teaching people how to program, and I watch as people go from desperately trying to solve problems by blindly adapting example code that they don't understand to actually thinking and being able to translate their thoughts into working, understandable programs. I think the key of it is to be thrust into situations that require understanding instead of just guessing the teacher's password -- the search space is too big for brute force. The class is all hands-on, doing toy problems that keep people struggling near the edge of their ability. And it works, somehow! I'm always amazed when they actually, truly learn something. I think this habit of expecting to understand things can be taught in at least one field, albeit painfully.

Is this something that people can learn in general? How? I consider this a hugely important question.

Comment author: John_Maxwell_IV 21 January 2010 07:31:49AM 6 points [-]

I wouldn't be surprised if thinking this way about computer programs transfers fairly well to other fields if people are reminded to think like programmers or something like that. There are certainly a disproportionate number of computer programmers on Less Wrong, right?

Comment author: wedrifid 21 January 2010 07:44:37AM 5 points [-]

There are certainly a disproportionate number of computer programmers on Less Wrong, right?

And those that aren't computer programmers would display a disproportionate amount of aptitude if they tried.

Comment author: John_Maxwell_IV 21 January 2010 09:08:25AM 9 points [-]

Certainly; I think this is a case where there are 3 types of causality going on:

  1. Using Less Wrong makes you a better programmer. (This is pretty weak; for most programmers, there are probably other things that will improve your programming skill a hundred times faster than reading Less Wrong.)
  2. Improving as a programmer makes you more attracted to Less Wrong.
  3. Innate rationality aptitude makes you a better programmer and more attracted to Less Wrong. (The strongest factor.)
Comment author: TheOtherDave 22 November 2010 01:33:32AM 2 points [-]

Is there any way to get someone to start expecting reality to make sense?

If I want to condition someone into applying some framing technique T, I can put them in situations where their naive framing Fn obtains no reward and an alternate framing Fa does, and Fa is a small inferential step away from Fn when using T, and no framings easily arrived at using any other technique are rewarded.

The programming example you give is a good one. There's a particular technique required to get from a naive framing of a problem to a program that solves that problem, and until you get the knack of thinking that way your programs don't work, and writing a working program is far more rewarding than anything else you might do in a programming class.

Something similar happens with puzzle-solving, which is another activity that a lot of soi-disant rationalists emphasize.

But... is any of that the same as getting people to "expect reality to make sense"; is it the same as that "click" the OP is talking about? Is any of it the same as what the LW community refers to as "being rational"?

I'm not sure, actually. The problem is that in all of these cases the technique comes out of an existing scenario with an implicit goal, and we are trying to map that post facto to some other goal (rationality, click, expecting reality to make sense).

The more reliable approach would be to start from an operational definition of our goal (or a subset of our goal, if that's too hard) and artificially construct scenarios whose reward conditions depend on spanning inferential distances that are short using those operations and long otherwise... perhaps as part of a "Methods of Rationality" video game or something like that.

Comment author: ialdabaoth 31 December 2013 07:54:05PM 0 points [-]

I guess that if I tell myself, "Most people simply do not expect reality to make sense, and are trying to do entirely different things when they engage in the social activity of talking about it", then I do feel a little less confused.

This is a testable hypothesis. To test it, see how deeply wise you appear when explaining to people who seem crazy that everything actually does have an underlying explanation, and then giving a quick and salient example.

Comment author: Aurini 21 January 2010 03:57:30AM 31 points [-]

During my military radio ops course, I realized that the woman teaching us about different frequencies literally thought that 'higher' frequencies were higher off the ground. Like you, I found her explanations deeply confusing, though I suspect most of the other candidates would have said it made sense. (Despite being false, this theory was good enough to enable radio operations - though presumably not engineering).

Thankfully I already had a decent founding in EM, otherwise I would have yet more cached garbage to clear - sometimes it's worse than finding the duplicate mp3s in my music library.

Comment author: bogus 21 January 2010 08:02:52PM *  4 points [-]

Our trainer was explaining something (the tracert command) but I didn't understand it because his explanation didn't seem to make sense.

Could you clarify? To properly understand how traceroute works one would need to know about the TTL field in the IP header (and how it's normally decremented by routers) and the ICMP TTL Exceeded message. But I'm not sure that a tech support drone would be expected to understand any of these.

Comment author: arundelo 22 January 2010 12:13:17AM *  25 points [-]

To properly understand how traceroute works one would need to know about the TTL field

I did learn about this on my own that day, but the original confusion was at a quite different level: I asked whether the times on each line measured the distance between that router and the previous one, or between that router and the source. His answer: "Both." A charitable interpretation of this would be "They measure round trip times between the source and that router, but it's just a matter of arithmetic to use those to estimate round trip times between any two routers in the list" -- but I asked him if this was what he meant and he said no. We went back and forth for a while until he told me to just research it myself.

Edit: I think I remember him saying something like "You're expecting it to be logical, but things aren't always logical".

Comment author: denisbider 25 January 2010 03:43:52PM 30 points [-]

Jesus Christ. "Things aren't always logical." The hallmark of a magic-thinker. Of course everything is always logical. The only case it doesn't seem that way is when one lacks understanding.

Comment author: MichaelGR 20 January 2010 11:53:05PM 1 point [-]

Looks like he was just repeating various teacher's passwords.

Comment author: AndyWood 20 January 2010 07:24:01PM *  25 points [-]

This is overwhelmingly how I perceive most people. This in particular: 'reality is social'.

I have personally traced the difference, in myself, to receiving this book at around the age of three or four. It has illustrations of gadgets and appliances, with cut-out views of their internals. I learned almost as soon as I was capable of learning, that nothing is a mysterious black box, things that seem magical have internal detail, and there are explanations for how they work. Whether or not I had anything like a pre-existing disposition that made me love and devour the book in the first place, I still consider it to have had a bigger impact on my whole world view than anything else I can remember.

Comment author: VAuroch 31 December 2013 07:35:51PM 1 point [-]

I got Macaulay's The Way Things Work (the original) at a slightly higher age. I suspect a big reason I became a computer scientist was the joy of puzzling through the adder diagrams and understanding why they worked.

Comment author: Nisan 23 January 2014 11:35:10PM 0 points [-]

I traced those adder diagrams as a child as well, and it surely was a formative experience.

Comment author: FourFire 23 January 2014 09:14:34PM 1 point [-]

This is mine which I recieved at around age six. I don't recall how many tens of times I read and reread those pages.

Comment author: Stuart_Armstrong 20 January 2010 07:54:51PM 19 points [-]

This is worth an entire post by itself. Cheers.

Comment author: wedrifid 20 January 2010 11:16:40PM 5 points [-]

Yes, please!

Comment author: gwern 20 January 2010 08:07:11PM 13 points [-]

So I called him back and had a little chat about it. The idea that the project had succeeded because I designed it that way had not occurred to him, and the idea that I had done it by the way I negotiated the requirements in the first place -- as opposed to heroic efforts during the project -- was quite an eye opener for him.

The Inside View says, 'we succeeded because of careful planning of X, Y, and Z, and our own awesomeness.' The Outside View says, 'most large software projects fail, but some succeed anyway.'

Comment author: pjeby 20 January 2010 08:47:01PM 14 points [-]

The Inside View says, 'we succeeded because of careful planning of X, Y, and Z, and our own awesomeness.' The Outside View says, 'most large software projects fail, but some succeed anyway.'

What makes you think it was the only one, or one of a few out of many?

The specific project was only relevant because my bosses prior to that point in time already implicitly understood that there was something my team was doing that got our projects done on time when others under their authority were struggling - but they attributed it to intelligence or skill on my part, rather than our methodology/philosophy.

The newer boss, OTOH, didn't have any direct familiarity with my track record, and so didn't attribute the success to me at all, except that obviously I hadn't screwed it up.

Comment author: thomblake 20 January 2010 07:04:37PM 4 points [-]

Not usually a fan of your thoughts, but these seem right on the money.

Comment author: gwern 10 October 2010 03:00:46AM 1 point [-]
Comment author: Sniffnoy 21 January 2010 08:33:58AM 1 point [-]

(Side note: I find myself often trying to find a way to express grasp/control as a pair, because really the two are the same. If you really grasp something, you should be able to control it, at least in principle.)

Well, anything mathematical would be an exception to that, at the least.

Comment author: pjeby 22 January 2010 02:28:03AM 4 points [-]

Well, anything mathematical would be an exception to that, at the least.

If you really grasp something mathematical, you ought to be able to apply it -- at least in principle.

Comment author: Sniffnoy 22 January 2010 09:01:35AM 1 point [-]

OK but that's not really what "control" normally means, is it? "Manipulate" might be a better word here.

Comment author: RobinZ 22 January 2010 12:43:36PM 1 point [-]

"Manipulate" would also extend the thinking-as-holding metaphor of "grasp".

(I have to admit that I was confused by "control" as well.)

Comment author: Gunnar_Zarncke 05 August 2014 05:34:32AM 0 points [-]

reality is social

thus the concept of gods.