LESSWRONG
LW

Comment Permalink
jimrandomh20

There are reference classes of thoughts which one shouldn't let pass without some minimum amount of follow-up thinking: high-value opportunities, credible dangers, and implicit decisions recognized for the first time, to name a few examples. It's good to acknowledge that these exist, to have a detector to catch them as they pass by, and to throw an error signal if your mind tries to move onto a new topic while a valuable line of thought is still unresolved.

Reply
See in context
by Dorikka
2 min read
38

13

In Anna’s recent post, she talked about training your mind to notice when it wasn’t curious about something and scream “Error! Look for a different way to do this” in such cases. Johnicholas and TheOtherDave's list of what stupidity feels like also looks useful for this purpose. I'm creating this post to make a more comprehensive list of feelings which indicate that people should reanalyze different possible paths to make sure that the one which they're taking is the most effective one to their objective.

Please suggest additions to the list in your comments -- I'll move them up here (along with links to further explanation, if given.) Keep in mind that your description of the feeling should be as illustrative as possible. For example, "feeling stupid" is unhelpful, while "you feel like you've taken a wrong turn into a never-ending tunnel" is better. Of course, metaphors which are immediately understood by some people may not be so easily understood by others, so try to give a more detailed description of the feeling if other people express that you're probably saying more than they're hearing.

List: "Error! Look for a different way to do this" if you feel like:

  • being bored, being in pain, being distracted, wanting to do anything else than this
  • being unworthy of these divine (external) ideas
  • blind plodding obedience
  • being tired all the time, even if you're not2
  • not having enough fingers to hold all of my thoughts in place
  • merging onto the highway when I can't see all the oncoming traffic
  • someone's playing loud distracting music that I can't hear
  • riding on a train with square wheels
1. Sometimes tedious/boring tasks genuinely cannot be made easier or less boring, so your "Error!" message might not return anything useful. However, you should at least look.
2. This may also indicate that your stupidity has biological causes, such as nutrition/sleep deficiency. 20-30 minute naps are awesome, though longer ones might make you groggy.
3. Of course, if a goal-achieving action is also supported by authorities, that is a good thing.
Personal Blog

13

Mentioned in
When to scream "Error!"
New Comment
38 comments, sorted by
top scoring
Click to highlight new comments since:
[-]TheOtherDave80

An error signal I often watch out for is feeling like I have to avoid working out the implications of a particular line of reasoning. It is generally a sign that I fear it constitutes evidence against a belief I'm emotionally invested in.

Another is recognizing when I'm privileging my own position... that is, when I believe X, but I'm aware that if the accidental circumstances of my life were different I would believe -X. That's not always a problem -- sometimes my position genuinely is superior! -- but it's a warning sign.

Reply
[-]Oligopsony00

Another is recognizing when I'm privileging my own position... that is, when I believe X, but I'm aware that if the accidental circumstances of my life were different I would believe -X. That's not always a problem -- sometimes my position genuinely is superior! -- but it's a warning sign.

I agree that something like this has to be an important warning sign. But how do I calibrate this sense so that it isn't going off all the time (and thus no longer serving as evidence of anything?) Even if we limit it to situations where the "different circumstances" aren't "having access to more or less relevant information, and the question at hand is non-normative, that's still a vast space of disagreement that goes dingdingding! for almost any nontrivial question. I can of course notice when my opinions are atypical for, say, young college-educated white men, but even then it seems likely that some not-as-quantifiable aspect of my experiences, other than the relevant evidence I've gathered or could infer[1], may have led me to this. It's certainly very easy to construct stories to this effect!

Perhaps demographic atypicality of opinions serves as Bayesian evidence that your views do just happen to be the result of (good) epistemic luck, and are therefore true?

[1] i.e., if I never read in detail the arguments and theories of those with very different ideologies than my own, I must assume that they're at least somewhat convincing, as they're convincing to someone. Preservation of expected evidence, &c.

Reply
[-]TheOtherDave10

Say I'm in circumstance X and believe Y, and you're in X' and believe Y'..

I can try to understand what evidence you have for Y' -- that is, try to understand the subset of your experience of X' that is relevant to believing Y'. I can then use that understanding as additional evidence that informs my estimates of the likelihood of Y and Y'.

Or I can decide that I don't feel like putting that much effort into the question. Which is perfectly valid... as you say, this comes up all the time, and one has to prioritize.

In that second case, I can take various shortcuts.

Indeed, many such shortcuts are wired into my brain, and they aren't necessarily bad ones -- though of course experienced deceivers are used to subverting them. Others can be learned, either implicitly or explicitly. Some are so unreliable in the modern world I do best to _un_learn them.

What I try not to do is fool myself into thinking I've actually evaluated the situation when I've merely taken a shortcut. If I'm dismissing Y' and reaffirming my belief in Y without doing the analysis, the alarm goes off to remind me that no, that's not justified. Y is my current belief, and I'm choosing not to investigate Y' because I've got better things to do, and that's really all I can legitimately say.

Reply
[-]JGWeissman60

It is useful to have a black list of problem indicators, if you can train yourself so these indicators jump out at you. But also approach this problem from the other side, building a white list of good cognitive states. When should you not scream "Error!"?

Reply
[-]jimrandomh50

But also approach this problem from the other side, building a white list of good cognitive states. When should you not scream "Error!"?

Some noteworthy cases come to mind:

  1. When concluding that a decision you made more than a minute ago was a mistake (because the error signal will be misattributed to the investigation, rather than the mistake).
  2. Within the first minute of starting a task (because it will attach to the act of starting and make you a procrastinator)
  3. When evaluating statements that are especially emotionally salient (because the error signal is more likely to correspond to unpleasant conclusions than to falsehoods)
  4. When noticing that your brain has screamed Error! at you (because you need to clear that signal before evaluating the original error)
Reply
[-]JGWeissman20

You seem to have interpreted my question to mean "When do you scream 'Error!' when you shouldn't?". While your response contains valuable answers to that question, what I had in mind was building a (nearly) comprehensive list of indicators that you are currently being an effective rationalist, so that when you don't notice any good indicator, you can scream "Error!".

Reply
[-]Dorikka00

I treat not screaming "Error!" as the default state, and I think that you should at least pay attention to problem indicators even if you otherwise feel like you're grokking well, as doing so might help to prevent you from going down the wrong mental path.

Do you have examples of white list states, where you should be more prone to screaming "Error!" if you don't have many/any of them?

Reply
[-]JGWeissman10

I treat not screaming "Error!" as the default state, and I think that you should at least pay attention to problem indicators even if you otherwise feel like you're grokking well, as doing so might help to prevent you from going down the wrong mental path.

I am not dismissing your approach. I am saying do both.

Do you have examples of white list states, where you should be more prone to screaming "Error!" if you don't have many/any of them?

My first example is that you are pursuing a worthwhile goal, which is an attempt at refining Anna's "use curiosity" heuristic. (Notice how even though Anna presented it as lack of curiosity is a negative indicator, I see it as curiosity is a positive indicator.)

Reply
[-]Dorikka00

Okay, that makes sense to me. However, I think that our two lists are working in two different domains. I think that mine is best at determining whether you could be more effective in achieving a goal while yours is better at determining whether your current actions line up with more terminal goals.

Reply
[-]David_Gerard50

It strikes me that a lot of cognitive behavioural therapy involves catching yourself in the process of particular thoughts, thought patterns or feelings. My girlfriend has recently completed a course of CBT and the change in her has been amazing. So many little irrationalities have stopped happening, and she was pretty clear-thinking before that!

So, I have here a copy of "Cognitive Behavioral Therapy For Dummies". The "For Dummies" books have a reputation for high quality :-) I'll see if there's useful tidbits in there for thought-spotting.

Reply
[-]jimrandomh30

Stupidity feels like: [...] being tired all the time, even when I'm not

That's not stupidity, that's unhealthiness. Which can and often does cause stupidity, but it's not the same thing, and it can't be fixed in the same ways.

Reply
[-]Dorikka00

I agree with you, so I'm going to tentatively remove it. TheOtherDave, this one was from your comment -- if you think that it describes genuine stupidity rather than unhealthiness, could you explain why?

Reply
[-]TheOtherDave20

Admittedly, I don't have a lot of personal experience with healthy stupidity.

I'm a fairly bright guy, by human standards, and most of my experience with genuine stupidity stems from a period of my life where I was recovering from stroke-induced brain damage, which is at least arguably a form of unhealthiness.

That said, I think everyone is familiar with the effects of fatigue on cognition. It's harder to hold onto a train of thought, it's harder to account for emotional bias, it's harder to keep things in memory. When I get tired, I get stupid. It's a kind of mental fog.

When I was recovering, I felt like that all the time.

Partly, of course, that's because I was genuinely tired a lot of the time, in large part because my blood pressure was being artificially lowered and I was greying out a lot.

But even when I wasn't tired -- during those brief blessed windows after waking up and before I started physical therapy, for example -- there was still that characteristic mental fog, all the time.

That's what I was thinking about, when I wrote it.

That said, I have no idea whether healthy stupidity feels at all like that.

Reply
[-]Dorikka10

Okay. Considering that you were genuinely tired during the time period about which this observation was made (and that your tiredness probably facilitated stupidity, rather than the other way around), this does seem like an indicator of tiredness rather than stupidity, so I'm going to leave it out for now. Thanks for the explanation.

Reply
[-]jimrandomh10

Actually, on further consideration, I'd be inclined to leave it in, with a note to the effect that stupidity with biological causes is often curable. I don't know what fraction of stupidity is caused by health problems, but it definitely doesn't take anything nearly so dramatic as a stroke to cause mind fog, and I suspect it would rank highly on any cause-of-stupidity diagnostic checklist. In particular, subtle dietary deficiencies do it too.

Reply
[-]Dorikka00

Done. I note that I just reversed a decision twice, and might have made the correct decision quicker had I tried harder for goal-related insight before replying.

Adding "feeling like you're blind and must be led around by more observant people"

Reply
[-]jimrandomh20

There are reference classes of thoughts which one shouldn't let pass without some minimum amount of follow-up thinking: high-value opportunities, credible dangers, and implicit decisions recognized for the first time, to name a few examples. It's good to acknowledge that these exist, to have a detector to catch them as they pass by, and to throw an error signal if your mind tries to move onto a new topic while a valuable line of thought is still unresolved.

Reply
[-]Dorikka20

If you think this should be a top level post, upvote this comment and downvote the child.

Reply
[-]David_Gerard30

And, of course, upvote the post itself ;-)

Reply
[-]ShardPhoenix10

We really need proper polls. This one seems to have gone rather haywire.

Reply
[-]Normal_Anomaly00

If you think this shouldn't be a top-level post, upvote this and downvote the child.

Reply
[-]Normal_Anomaly-20

This is the child. Downvote it for karma balance iff you don't want Dorrika's post to be top-level.

Reply
[-]Normal_Anomaly10

...and the parent and grandparent got un-synched again. The point of the karma balance post is so that its karma is the opposite of the poll's karma. When you downvote one of a poll/karma balance pair, you upvote the other.

Reply
[-]wedrifid30

The point of the karma balance post is so that its karma is the opposite of the poll's karma.

It is a karma sink, not a balancer that must be equal but opposite. It is a way to short out people's desires to not reward karma for privilege of providing information, or a way to penalize the poll itself without damaging the data. Since we wouldn't gain information by having two copies of the same number there is no real need for the karma sink in sync.

What does seem to be a problem here is that your poll comment is (or rather was) negative. Your poll comment should never get a downvote, ever, if people are cooperating in the poll game. Downvoting defeats the purpose of having two separate comments for 'be top level' and "don't be top level". This just means that the downvoter has chosen not to submit to demands (or acquiesced with the request) to vote according to the prescribed pattern for these particular comments. They aren't cooperating within the poller's game. Unfortunately it is not possible to know how many people have just chosen not to play so all such votes corrupt the data rather significantly.

This is where having an explicit poll mechanic would be rather handy.

Reply
[-]Richard_Kennaway10

I find that the angrier I get, the stupider I get. Especially if I'm angry with an inanimate object, which is a stupidity in itself.

Reply
[-]David_Gerard10

When you get that feeling that what you're doing is "like watching cable, only with fewer hair replacement infomercials"[1]. It's noticing the particular kind of boredom you get when you're slipping into a not very interested consumerist attitude.

I have generally treated this as a signal to stop bothering with whatever it is. Could do with strategies for finding more interesting new things to do.

(I find myself uninterested in trying to get more interested in the not-sufficiently-interesting thing, and in fact dislike the idea of doing so. I'm not entirely clear why. Possibly it's a slight feeling of resentment at the whatever-it-is for wasting the seconds of my life. There is an inexhaustible supply of things that would just love one's precious attention, and I feel a need to be harsh in culling them.)

[1] Dunn, Sarah. The Official Slacker Handbook. Abacus, 1994. ISBN 0-349-10591-X

Reply
Moderation Log
38Comments