Kaj_Sotala comments on The Substitution Principle - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (64)
You are making many unanalyzed assumptions here.
1) You are assuming that your mind did or did not do certain things in those moments when it was quietly answering the question. In particular, you make assumptions like "....your answer probably didn't take any of the above complications into account. It's as if your brain, while generating an answer, never even considered them .....". Why are you so sure that it probably didn't take any of the above considerations into account?
To illustrate how wrong this might be, consider that when a cognitive psychologist gives someone a visual priming task, with (for example) masked cues, the subject reports that she did not take ANY account of the masked cues. And yet, she shows clear evidence that she did very much take the cue into account! The proof is right there in the reaction times (which depend on what was in the cue).
So if someone can be that wrong in their self-report assessment of what factors they are taking into account in a situation as siimple as masked priming, what is the chance that a person in one of the scenarios you describe above is also doing all kinds of assessments that actually happen below the reporting threshold? At the very least this seems likely. But even if you don't accept that it is likely, you still have to give reasons why we should believe that it is not happening.
So, when Kahneman cites substitutions, his evidence clearly distinguishes substitutions from complex assessments that may be interpreted as substitutions, or which are correlated with substitutions? I don't buy that.
My second objection has to do with the oversimplification of the analysis:
2) You seem to be framing a lot of scenarios as if they were all instances of the same type of problem. As if the same mechanism was operating in most or all of these circumstances. Your mechanism involves a "target question", a "substituted question" (assumed to be of dubious validity) and a resulting answer that is assumed to be of sub-optimal quality. While there may be some situations where this frame neatly applies to a situation, I do not believe that it applies to all, and nor do I believe that it helps to try to oversimplify all instances of "bias" so they can be squeezed into this narrow frame.
At the very least, there appear to be situations that do not fit the pattern. Chess skill, for example. The question asked by the chess player is "How do I take the opponent's King?". But rather than address this question directly (as I did, in my very first chess game, when I imagined a sequence of moves that culminated in me taking that King, then started executing my planned sequence of moves .....), the expert chess player knows that a different set of questions have to be asked: to wit, "How do I make pleasing, coherent patterns of support and strength on the board?" and "Do I recognize anything about the current pattern as similar or identical to a situation I have seen in the past?"
That particular "substitution" happens to be extremely optimal. It also happens to be not how chess computers work (by and large: let's not get sidetracked by the finer points of chess programming ... the fact is that machines rely on depth to a very large extent, whereas humans rely on pattern). So it makes no sense to talk about substitution as a "problem" in this case. Far from it, substitution seems to be why a tiny little lookahead device (human mind) can give the massive supercomputer a run for its money.
3) Finally, this analysis overall has the feel (like almost all "human bias is a problem" arguments) of fitting the data to the theory. So many people (here on LW, and in the biasses community generally) want to see "Biasses" as a big deal, that they see them everywhere. All the evidence that you see supports the idea that the concept of "biasses and heuristics" is a meaningful one, and that there are many instances of that concept, and that thos instances have certain ramifications.
But, like people who see images of Jesus in jars of Marmite, or evidence of divine intervention whenever someone recovers from an illness, I think that you see evidence of bias (and, in this case, substitution) primarily because you have trained yourself to theorize about the world that way. Not because it is all real.
This is a fair criticism, and you're right - I can't say with definite certainty that these things were actually never considered at all. Still, if those things were considered, they don't seem to be reflected in the final output. If instead of sayng "the thing is, it probably didn't", I said "the thing is, it probably didn't - or if it did, it's difficult to notice from the provided answer", would you consider that acceptable?
I think you might be somewhat misinterpreting me here. I didn't say that substitution is necessarily a problem - I specifically said it probably works pretty well most of the time. Heck, I imagine that if I was building an AI, I would explicitly program something like a substitution heuristic into it, to be used most of the time - because difficult problems are genuinely difficult, both in the sense of being computationally expensive and requiring information that isn't usually at hand. A system that always tried to compute the exact answer for everything would never get anything done. Much better to usually employ some sort of quick heuristic that tended to at least point in the right direction, and then only spend more effort on the problem if it seemed to be important.
For that matter, you could say that a large part of science consists of a kind of substitution. Does System 1 actually ignore all those complicated considerations when considering its answer? Well, we can't really answer that directly... but we can substitute the question with "does it seem to be ignoring them in certain kinds of experimental setups", and then reflect upon what the answers to that question seem to tell us. This is part of the reason why I felt confident in saying that the brain probably never did take all the considerations into account - because simplifying problems to easier ones is such an essential part of actually ever getting anything done that it would seem odd if the brain didn't do that.
So I agree that there are many cases (including your chess example) where substitution isn't actually a problem, but rather the optimal course of action. And I agree that there are also many cases of bias where the substitution frame isn't the best one.
The reason why I nevertheless brought it up was that, if I were building my hypothetical AI, there's still one thing that I'd do differently than how the human brain seems to do it. A lot of the time, humans seem to be completely unaware of the fact that they are actually making a substitution, and treat the answer they get as the actual answer to the question they were asking. Like I mentioned in my other comment, I think the substitution principle is useful because it gives a useful rule of thumb that we can use to notice when we might be mistaken, and might need to think about the matter a bit more before assigning our intuitive result complete confidence.