LESSWRONG
LW

Comment Permalink
Elo20

try this:

To make a judgment based on a not reliable feeling is a different mistake from experiencing a reliable feeling and drawing an incorrect conclusion through an error in judgment.

or presented like this:

the process of making a judgement on a feeling:
not reliable feeling -> judgement that ends up being wrong because of some unreliable feeling data
is not the same mistake as
reliable feeling -> judgement that is an incorrect judgement.

Reply
See in context
by Gram_Stone
3 min read
16

23

(This is mostly a summary of Evans (2012); the fifth misconception mentioned is original research, although I have high confidence in it.)

It seems that dual process theories of reasoning are often underspecified, so I will review some common misconceptions about these theories in order to ensure that everyone's beliefs about them are compatible. Briefly, the key distinction (and it seems, the distinction that implies the fewest assumptions) is the amount of demand that a given process places on working memory.

(And if you imagine what you actually use working memory for, then a consequence of this is that Type 2 processing always has a quality of 'cognitive decoupling' or 'counterfactual reasoning' or 'imagining of ways that things could be different', dynamically changing representations that remain static in Type 1 processing; the difference between a cached and non-cached thought, if you will. When you are transforming a Rubik's cube in working memory so that you don't have to transform it physically, this is an example of the kind of thing that I'm talking about from the outside.)

The first common confusion is that Type 1 and Type 2 refer to specific algorithms or systems within the human brain. It is a much stronger proposition, and not a widely accepted one, to assert that the two types of cognition refer to particular systems or algorithms within the human brain, as opposed to particular properties of information processing that we may identify with many different algorithms in the brain, characterized by the degree to which they place a demand on working memory.

The second and third common confusions, and perhaps the most widespread, are the assumptions that Type 1 processes and Type 2 processes can be reliably distinguished, if not defined, by their speed and/or accuracy. The easiest way to reject this is to say that the mistake of entering a quickly retrieved, unreliable input into a deliberative, reliable algorithm is not the same mistake as entering a quickly retrieved, reliable input into a deliberative, unreliable algorithm. To make a deliberative judgment based on a mere unreliable feeling is a different mistake from experiencing a reliable feeling and arriving at an incorrect conclusion through an error in deliberative judgment. It also seems easier to argue about the semantics of the 'inputs', 'outputs', and 'accuracy' of algorithms running on wetware, than it is to argue about the semantics of their demand on working memory and the life outcomes of the brains that execute them.

The fourth common confusion is that Type 1 processes involve 'intuitions' or 'naivety' and Type 2 processes involve thought about abstract concepts. You might describe a fast-and-loose rule that you made up as a 'heuristic' and naively think that it is thus a 'System 1 process', but it would still be the case that you invented that rule by deliberative means, and thus by means of a Type 2 process. When you applied the rule in the future it would be by means of a deliberative process that placed a demand on working memory, not by some behavior that is based on association or procedural memory, as if by habit. (Which is also not the same as making an association or performing a procedure that entails you choosing to use the deliberative rule, or finding a way to produce the same behavior that the deliberative rule originally produced by developing some sort of habit or procedural skill.) When facing novel situations, it is often the case that one must forego association and procedure and thus use Type 2 processes, and this can make it appear as though the key distinction is abstractness, but this is only because there are often no clear associations to be made or procedures to be performed in novel situations. Abstractness is not a necessary condition for Type 2 processes.

The fifth common confusion is that, although language is often involved in Type 2 processing, this is likely a mere correlate of the processes by which we store and manipulate information in working memory, and not the defining characteristic per se. To elaborate, we are widely believed to store and manipulate auditory information in working memory by means of a 'phonological store' and an 'articulatory loop', and to store and manipulate visual information by means of a 'visuospatial sketchpad', so we may also consider the storage and processing in working memory of non-linguistic information in auditory or visuospatial form, such as musical tones, or mathematical symbols, or the possible transformations of a Rubik's cube, for example. The linguistic quality of much of the information that we store and manipulate in working memory is probably noncentral to a general account of the nature of Type 2 processes. Conversely, it is obvious that the production and comprehension of language is often an associative or procedural process, not a deliberative one. Otherwise you still might be parsing the first sentence of this article.

Personal Blog

23

Common Misconceptions about Dual Process Theories of Human Reasoning
New Comment
16 comments, sorted by
top scoring
Click to highlight new comments since:
[-]SquirrelInHell40

Oh man.

I didn't know there were "respectable" sources that basically make the same points I found out by myself.

I mentioned all what you wrote (except the fifth issue), and some more, in this video: https://www.youtube.com/watch?v=TV8sabfHelQ (warning, bad English accent)

The fifth common confusion is that, although language is often involved in Type 2 processing, this is likely a mere correlate

I've had a similar observation on my own after experimenting with wordless thinking.

I found that I could think wordlessly and imagelessly, and even have new insights while doing that, but I could not store those abstract new insights in short-term memory without vocalizing them.

Reply
[-]Gram_Stone20

It seems to sacrifice precision for accessibility, but I still like it very much. Also, you definitely have a thick accent, but you can certainly still be said to enunciate clearly, and I perfectly understood everything that you said. Also, I detected no syntactic or semantic errors, and your diction sounds very high.

Reply
[-]Kaj_Sotala30

The fourth common confusion is that Type 1 processes involve 'intuitions' or 'naivety' and Type 2 processes involve thought about abstract concepts. You might describe a fast-and-loose rule that you made up as a 'heuristic' and naively think that it is thus a 'System 1 process', but it would still be the case that you invented that rule by deliberative means, and thus by means of a Type 2 process. When you applied the rule in the future it would be by means of a deliberative process that placed a demand on working memory, not by some behavior that is based on association or procedural memory, as if by habit.

I suspect that we're disagreeing on the definitions of words rather than having any substantial difference in expectations, but: I think the way "intuition" is commonly used refers to vague feelings that you can't quite justify explicitly, not explicit heuristics that you've generated by deliberation. So your example doesn't really feel like a counterexample for the claim that intuitions are a Type 1 process.

Reply
[-]Gram_Stone20

Here are my ideological Turing test results of your comment:

People usually use the word intuition to refer to vague impressions that are not amenable to the same sort of justification as deliberative judgments, so these are different from the example that you provided of quickly inventing a deliberative rule and making errors in the process. This makes the purported counterexample less persuasive to me than you seem to expect it to be. Evaluate this comment in the context that we both still anticipate the same experiences, so this is likely a disagreement over word usage, and not likely to be highly significant.

I think this is a very productive criticism. I feel emphasis in italics makes it easier for me to write because it makes it more similar to the way that I speak, so please don't interpret them as aggressive. The way my mind goes down this path is thus:

I have to make the qualification that I don't believe that intuitions are vague feelings that cannot be justified, but vague feelings that have not been justified. There is always some fact of the matter as to whether or not it is, in some sense. But once again, probably something we would consider as disagreeing about word usage. But I think it's an important boundary to draw. From Evans (2006):

If intuition means based on feelings without access to explicit reasoning, then that sounds like a type 1 process. But in some applications it seems to mean naïve judgement, which could be based on explicit rules or heuristics that occur to an untrained judge, in which case they would be type 2.

People often use the phrase 'intuition' to refer to confident beliefs retrieved from cached memory, and the idea is that when you go wrong, it's because intuitions are unreliable. I'm getting at the possibility that that's what people say, but it's not the whole picture.

Say that you're a judge on Pop Idol or something like that, and you have no experience doing it, and you want to quickly come up with a rule, and you retrieved the reliable intuition that pop idols are usually very physically attractive, and then invented a deliberative rule that used your subjective rating of each candidate's physical attractiveness as a measure for evaluating their general Pop Idol factor, and suppose that physical attractiveness actually does not correlate perfectly with the true general Pop Idol factor. Then you would have begun with a reliable intuition and put it into an unreliable deliberative process and obtained an 'unreliable' result in the sense that it does not optimize for the purported normative criterion of Pop Idol judgment panels, which is the selection of the best Pop Idol; you would have picked the most attractive candidate instead, and you would have made a mistake on a higher level than using an unreliable intuition: you would have combined reliable intuitions in a deliberative but unreliable way. This is closely related to the 'System 1 is fast, System 2 is slow' distinction. Reasoning that looks like fast, unreliable intuitive reasoning can really just be fast, unreliable deliberative reasoning. So the main point is not about saying that there are a lot of counterexamples to 'intuitive' reasoning being System 1, but that if you want to do real work the category 'intuitive' won't cut it, because it's still a leaky generalization, even if it isn't that leaky. Does that all make sense?

Reply
[-]Kaj_Sotala30

I liked your rephrasing of my comment. :) I felt that it was an accurate summary of what I meant.

I believe that we're in agreement about everything.

Reply
[-][anonymous]00

Here are my ideological Turing test results of your comment:

People usually use the word intuition to refer to vague impressions that are not amenable to the same sort of justification as deliberative judgments, so these are different from the example that you provided of quickly inventing a deliberative rule and making errors in the process. This makes the purported counterexample less persuasive to me than you seem to expect it to be. Evaluate this comment in the context that we both still anticipate the same experiences, so this is likely a disagreement over word usage, and not likely to be highly significant.

I think this is a very productive criticism. I feel emphasis in italics makes it easier for me to write because it makes it more similar to the way that I speak, so please don't interpret them as aggressive. The way my mind goes down this path is thus:

I have to make the qualification that I don't believe that intuitions are vague feelings that cannot be justified, but vague feelings that have not been justified. There is always some fact of the matter as to whether or not it is, in some sense. But once again, probably something we would consider as disagreeing about word usage. But I think it's an important boundary to draw. From Evans (2006):

If intuition means based on feelings without access to explicit reasoning, then that sounds like a type 1 process. But in some applications it seems to mean naïve judgement, which could be based on explicit rules or heuristics that occur to an untrained judge, in which case they would be type 2.

People often use the phrase 'intuition' to refer to confident beliefs retrieved from cached memory, and the idea is that when you go wrong, it's because intuitions are unreliable. I'm getting at the possibility that that's what people say, but it's not the whole picture.

Say that you're a judge on Pop Idol or something like that, and you have no experience doing it, and you want to quickly come up with a rule, and you retrieved the reliable intuition that pop idols are usually very physically attractive, and then invented a deliberative rule that used your subjective rating of each candidate's physical attractiveness as a measure for evaluating their general Pop Idol factor, and suppose that physical attractiveness actually does not correlate perfectly with the true general Pop Idol factor. Then you would have begun with a reliable intuition and put it into an unreliable deliberative process and obtained an 'unreliable' result in the sense that it does not optimize for the purported normative criterion of Pop Idol judgment panels, which is the selection of the best Pop Idol; you would have picked the most attractive candidate instead, and you would have made a mistake on a higher level than using an unreliable intuition: you would have combined reliable intuitions in a deliberative but unreliable way. This is closely related to the 'System 1 is fast, System 2 is slow' distinction. Reasoning that looks like fast, unreliable intuitive reasoning can really just be fast, unreliable deliberative reasoning. Does that all make sense?

[This comment is no longer endorsed by its author]Reply
[-]Elo30

the mistake of entering a quickly retrieved, unreliable input into a deliberative, reliable algorithm is not the same mistake as entering a quickly retrieved, reliable input into a deliberative, unreliable algorithm.

quite a mouthful - can you simplify this maybe?

Reply
[-][anonymous]20

If I understand correctly, Gram_Stone is using input here to refer to the organism's short term memory, and reliable algorithm to refer to working memory (unreliable algorithm means no use of working memory), and output to refer to the organism's observable behavior. I'm not entirely certain of how he's distinguishing between input, algorithm, and output in this context, but that's my best guess.

The paragraph is a criticism of certain varieties of "thinking fast, thinking slow" sorts of arguments.

A common problem within psychology research is that researchers will conduct a study in which the independent variable is a stimulus in the external environment and the dependent variable is the organism's behavior, but then the researcher will make a claim about the underlying cognitive process without any evidence to make that claim. Without collecting data on the process, such studies are frequently not able to distinguish between different parts of the cognitive process.

So it is not possible to determine from examining an unreliable output whether that lack of reliability was due to an unreliable input or due to an unreliable algorithm.

The key thing to note is that the input in that sentence is not referring to the independent variable. The independent variable is being manipulated in a step before the input.

This was really helpful though. Dual-processing theory has always come across to me as being all over the map in terms of the definition of what is type 1 and what is type 2.

Reply
[-]Elo10

I do get what it means; (and you helped as well with your extended explanation) but I was more interested in encouraging him to word it in a way that was clearer and less of a mouthful (in the top post) to read over. In the hope that more people might engage with it if it can be explained more simply; which I think it can; (in this comment http://lesswrong.com/lw/nf5/common_misconceptions_about_dual_process_theories/d6oj)

Reply
[-]Gram_Stone00

short term memory

It could be long-term memory too.

This was really helpful though. Dual-processing theory has always come across to me as being all over the map in terms of the definition of what is type 1 and what is type 2.

Thanks. This is because anyone can call their theory a dual process theory. And it's an even more general term in all of psychology, all LWers are secretly talking about the enormous subset of dual process theories of reasoning, which is why I made the title what it was. And about ten years ago they were in the 'listing common characteristics stage' instead of the 'finding neurological correlates' stage. Evans is one of the very authors who caused the confusion, but he also cleared it up later, and admitted his mistake. There are some especially virtuous scientists within the field of cognitive science.

Reply
[-]Gram_Stone20

The sentence immediately after it is a paraphrase of the sentence in your quotation:

To make a deliberative judgment based on a mere unreliable feeling is a different mistake from experiencing a reliable feeling and arriving at an incorrect conclusion through an error in deliberative judgment.

Is that also too complex?

Reply
[-]Elo20

try this:

To make a judgment based on a not reliable feeling is a different mistake from experiencing a reliable feeling and drawing an incorrect conclusion through an error in judgment.

or presented like this:

the process of making a judgement on a feeling:
not reliable feeling -> judgement that ends up being wrong because of some unreliable feeling data
is not the same mistake as
reliable feeling -> judgement that is an incorrect judgement.

Reply
[-]ChristianKl20

Could you talk about intentions? Do you think it's a misconception to think of system II as the intentional system, or do you consider it to be an accurate term?

Reply
[-]Gram_Stone00

Excellent question. It would take a lot to present all of the evidence, but working memory still seems to me to remain the best distinction, because the many things that you refer to as 'intention' also require working memory to function. Intentionality is probably not a bad term to use, but still noncentral I think.

Everything we want to talk about is in the prefrontal cortex. There are four major components worth distinguishing: lateral prefrontal cortex, frontal pole, medial frontal cortex, and ventromedial prefrontal cortex. The entire prefrontal cortex, but the lateral prefrontal cortex in particular, is a key component of working memory. This is evidenced by the poor performance of patients with neurological lesions in the prefrontal cortex on delayed-response tasks. Also, physiological studies on the actual cells in primates, and functional-imaging studies on humans, have shown that prefrontal cortex activation remains even after the original stimulus is absent. We are not quickly caching this information and constantly editing the cache, we are maintaining the information in a low-capacity, dynamic memory store, just like computers do with their RAM. All of the caching stuff is in other regions. There are also regions of the cortex responsible for goal-oriented behavior, which involve the generation of a hierarchy of goals and subgoals, and the aforementioned patients often also do poorly on this, for one, because they have difficulty imagining consequences (takes working memory and selection of information to infer consequences, i.e. counterfactual surgery) and also, because it seems they have a problem selecting task relevant information. So another big part of it is a selection mechanism that filters out the irrelevant parts of cached information, but notice that the only reason to do this is because you're trying to figure out whether or not you want it to take up space in working memory. Also notice that this failure at selecting relevant information is an explanation for why these patients have intact recognition memory but poor recency memory (ability to recognize how long ago they obtained some memory). Recognition memory is all cached data, recency memory requires you to select the parts of the cached data that are relevant to figuring out when the data was observed. I'm probably missing a lot of important distinctions and details still, so don't consider this the final word; see Chapter 13 of Gazzaniga, Ivry, and Mangun's Cognitive Neuroscience if you really want to get on top of this. Remarkably smooth read for the level of the content it contains.

Reply
[-]Petter00

Are these misconceptions really common? I thought Kahneman was pretty clear on this in Thinking, Fast and Slow.

Reply
[-]Good_Burning_Plastic20

He was. but it's not like everybody who ever uses his terminology has actually read his book.

Reply
Moderation Log
16Comments