The problem is that you assume that you know the relevant biases.

Wikipedia has a list; I've checked a few of them, and the rest are on my TODO list. I have that page watched so if there's a new bias I'll know.

There are often cases where you don't know why someone screws up. There are domains where it's easier to get knowledge about how much people screw up than understanding the reasons behind screwups.

Information is produced regardless, and often recorded (see e.g. Gwern's Mistakes page). So long as I myself don't screw up, which, assuming that I always follow my decision procedure and my decision procedure is correct, I won't, then it doesn't matter.

Fear produces fight or flight responses. People often fight out of fear. Aggression often comes out of weakness.

OK, but I was talking about "perceived willingness to be aggressive" (signal), not aggression (action).

A karate black belt is dominant but usually not aggressive. Taller people get payed more money because being tall is a signal for social dominance.

Someone wearing a black belt is probably going to be perceived as more aggressive, the same way someone idly cleaning their fingernails with a sharp knife might be. Similarly if a person adopts something recognized as a fighting stance. Not certain about tall people, that's probably something else besides perceived aggressiveness, e.g. "My parents were rich and could feed me a lot".

This has gone on long enough that it might be worth summarizing into a post... do you want to write it or should I?

Conflating whether or not you could do something to stop them with finding truth makes it harder to have an accurate view of whether or not the result is true. Accepting reality for what it is helps to have an accurate perception of reality.

I'm not certain where you see conflation. I have separate storage areas for things to think about, evidence, actions, and risk/reward evaluations. They interact as described here. Things I hear about go into the "things to think about" list.

Only once you understand the territory should you go out and try to change things.

The world is changing so I must too. If the apocalypse is tomorrow, I'm ready. I don't need to "understand" the apocalypse or its cause to start preparing for it. IF I learn something later that says I did the wrong thing, so be it. I prefer spending most of my time trying to change things than sitting in a room all day trying to understand. Indeed, some understanding can only be gained through direct experience. So I disagree with you here.

If you do the second step before the first you mess up your epistemology. You fall for a bunch of human biases evolved for finding out whether the neighboring tribe might attack your tribe that aren't useful for clear understanding of todays complex world.

The decision procedure I outlined above accounts for most biases; you're welcome to suggest revisions or stuff I should read.

I spoke about incentives. [...] The incentives go towards "spectual".

You didn't, AFAICT; you spoke about "inherent biases". I think my point still stands though; averaging over "all information everywhere" counteracts most perverse incentives, since perversion is rare, and the few incentives left are incentives that are shared among humans such as survival, reproduction, etc. In general humans are good at that sort of averaging, although of course there are timing and priming effects. Researchers/bloggers are incentivized to produce good results because good results are the most useful and interesting. Good results lead to good products or services (after a 30 year lag). The products/services lead to improved life (at least for some). Improved life leads to more free time and better research methods. And the cycle goes on, the end result AFAICT is a big database of mostly-correct information.

Scott H Young whom I respect and who's a nice fellow wrote his post against spaced repetition and still know recommends now in a later post the usage of Anki for learning vocabulary.

His post is entitled "Why Forgetting Can Be Good" and his mention of Anki is limited to "I’m skeptical of the value of an SRS for most domains of knowledge." If he then recommends Anki for learning vocabulary, this changes relatively little; he's simply found a knowledge domain where he found SRS useful. Different studies, different conclusions, different contributions to different decisions.

It's not about remembering it's about being able to make estimates even when you aren't sure.

You're never sure, so why mention "even when you aren't sure", since it's implied? Striking that out...

It's not about remembering it's about being able to make estimates.

Estimation comes after the evidence-gathering phase. If you have no evidence you can make no estimates. Fermi estimation is just another estimation method, so it doesn't change this. If you have no memory, then you have no evidence. So it is about remembering. "Those who cannot remember the past are condemned to repeat it".

And you can calibrate your error intervals.

If you have no estimates you can't have error intervals either. Indeed, you can't do calibration until you have a distribution of estimates.

Aggression is not the central word. Status and dominance also appear. People do a bunch of things to appear higher status.

It looks like the central word is definitely dominance. Stringing the top words into a sentence I get "Sports teams wear red to show dominance and it has an effect on referees' performance". I guess I was going off of the Mandrill story where signs of dominance are correlated with willingness to be aggressive. This study says dominance and threat are emphasized by wearing red, where "threat" is measured by "How threatening (intimidating, aggressive) did you feel?". Some other papers also relate dominance to aggressiveness. So I feel comfortable confusing the two, since they seem to be strongly correlated and relatively flexible in terms of definition.

The comments do focus on status, so I guess you have a point. But I generally skip over the comments when an article is linked to. And the status discussion was in the comments of Overcoming Bias post, so by no means central.

One of the studies in question suggested that it makes woman more attracted to you measured by the physical distance in conversation. Another one suggest that attraction based on photo ratings. I actually did the comparison on hotOrNot. I tested a blue shirt against a red shirt. Photoshopped so nothing besides the color with different. For my photo blue scored more attractive than red despite the studies saying that red is the color that raises attractiveness.

Would you be referring to, among others, this study? Unfortunately... it still looks like experimental psychology, so again I have to plead lack of statistics.

The replication rates for cancer biology seem to be even worse than for psychology if you trust the Amgen researchers who could only replicate 6 of 55 landmark studies that they tried to replicate.

I've mostly been reading Army / DoD studies, which have a different funding model. But I guess cancer will become relevant eventually (preferably later rather than sooner).

Side note: does LW have a "collapse threads more than N levels deep" feature like reddit? It probably should have triggered a few replies ago, so I didn't post on the wrong child...

For example, I don't want a red car because I don't want to get pulled over by the cops all the time.

The car story appears to be a myth nowadays, but that could just be due to the increased use of radar guns and better police training. Radar guns were introduced around the 1950's so all of their policemen quotes are too recent to tell.

I thought I had written all I could. What sort of things should I add?

Once upon a time I tried using what I could coin "quicklists". I took a receipt, turned it over to the back (clear side), and jotted down 5-10 things that I wanted to believe. Then I set a timer for 24 hours and, before that time elapsed, acted as if I believed those things. My experiment was too successful; by the time 24 hours were up I had ended up in a different county, with little recollection of what I'd been doing, and some policemen asking me pointed questions. (I don't believe any drugs were involved, just sleep deprivation, but I can't say for certain).

More recently, I rented and saw the film Memento, which explores these techniques in a fictional setting. The concept of short-term forgetting seemed reasonable and the techniques the character uses to work around it are easily adapted in real life. My initial test involved printing out a pamphlet with some dentistry stuff in tiny type (7 12-pt pages shrunk to fit on front-back of 1 page, folded in quarters), and carrying it with me to my dentist appointment. I was able to discuss most of the things from my pamphlet, and it did seem that the level of conversation was raised, but there were many other variables as well so it's hard to quantify the exact effect.

I'm not certain these techniques actually count as "doublethink", since the contradiction is between my "internal" beliefs and the beliefs I wrote down, but it does allow some exploration of the possibilities beyond rationality. I can override my system 2 with a piece of paper, and then system 1 follows.

NB: Retrieving your original beliefs after you've been going off of the ones from the paper is left as an exercise to the student

Humans are biased to overrate bad human behavior as a cause for mistakes.

If a crocodile bites off your hand, it's generally your fault. If the hurricane hits your house and kills you, it's your fault for not evacuating fast enough. In general, most causes are attributed to humans, because that allows actually considering alternatives. If you just attributed everything to, say, God, then it doesn't give any ideas. I take this a step further: everything is my fault. So if I hear about someone else doing something stupid, I try to figure out how I could have stopped them from doing it. My time and ability are limited in scope, so I usually conclude they were too far away to help (space-like separation), but this has given useful results on a few occasions (mostly when something I'm involved in goes wrong).

The decent thing is to orient yourself on whether similar studies replicate.

Not really, since the replication is more likely to fail than the original study (due to inexperience), and is subject to less peer-review scrutiny (because it's a replication). See http://wjh.harvard.edu/~jmitchel/writing/failed_science.htm. The correct thing to consider is followup work of any kind; for example, if a researcher has a long line of publications all saying the same thing in different experiments, or if it's widely cited as a building block of someone's theory, or if there's a book on it.

Regardless every publish-or-perish paper has an inherent bias to find spectacular results.

Right, people only publish their successes. There are so many failures that it's not worth mentioning or considering them. But they don't need to be "spectacular", just successful. Perhaps you are confusing publishing at all, even in e.g. a blog post, with publishing in "prestigious" journals, which indeed only publish "spectacular" results; looking at only those would give you a biased view, certainly, but as soon as you expand your field of view to "all information everywhere" then that bias (mostly) goes away, and the real problem is finding anything at all.

Let's say wearing red every day.

So the study there links red to aggression; I don't want to be aggressive all the time, so why should I wear red all the time? For example, I don't want a red car because I don't want to get pulled over by the cops all the time. Similarly for most results; they're very limited in scope, of the form "if X then Y" or even "X associate with Y". Many times, Y is irrelevant, so I don't need to even consider X.

Thinking that those Israeli judges don't give people parole because they don't have enough sugar in their blood right before mealtime. Going and giving every judge a candy before hearing every case to make it fair isn't warranted.

Sure, but if I'm involved with a case then I'll be sure to try to get it heard after lunchtime, and offer the judge some candy if I can get away with it.

That's fixable by training Fermi estimates.

You can memorize populations or memorize the Fermi factors and how to combine them, but the point stands regardless; you still have to remember something.

It's a reference to the controversy about whether washing your hands primes you to be more moral. It's a experimental social science result that failed to replicate.

Ah, social science. I need to take more courses in statistics before I can comment... so far I have been sticking to the biology/chemistry/physics side of things (where statistics are rare and the effects are obvious from inspection).

Exhibiting symptoms often considered as signs of mental illness. For example, this says 38.6% of general people have hallucinations. This says 40% of general people had paranoid thoughts. Presumably these groups aren't exactly the same, so there you go: between 0.5 and 0.8 of the general population. You can probably pull together some more studies with similar results for other symptoms.

Given replication rates of scientific studies a single study might not be enough.

Enough for what? My question is whether my hair stylist saying "Shaving makes the hair grow back thicker." is more reliable than http://onlinelibrary.wiley.com/doi/10.1002/ar.1090370405/abstract. In general, the scientists have put more thought into their answer and have conducted actual experiments, so they are more reliable. I might revise that opinion if I find evidence of bias, such as a study being funded by a corporation that finds favorable results for their product, but in my line of life such studies are rare.

Single studies that go against your intuition are not enough reason to update. Especially if you only read the abstract.

I find that in most cases I simply don't have an intuition. What's the population of India? I can't tell you, I'd have to look it up. In the rare cases where I do have some idea of the answer, I can delve back into my memory and recreate the evidence for that idea, then combine it with the study; the update happens regardless of how much I trust the study. I suppose that a well-written anecdote might beat a low-powered statistical study, but again such cases are rare (more often than not they are studying two different phenomena).

No need to get people to wash their hands before you do a business deal with them.

I wash my hands after shaking theirs, as soon as convenient. Or else I just take some ibuprofen after I get sick. (Not certain what you were trying to get at here...)

I don't have experience with those, but I'll recommend Graphviz as a free (and useful) alternative. See e.g. http://k0s.org/mozilla/workflow.svg

The simple answer is to ask someone else, or better yet a group; if D is small, then D^2 or D^4 will be infinitesimal. However, delusions are "infectious" (see Mass hysteria), so this is not really a good method unless you're mostly isolated from the main population.

The more complicated answer is to track your beliefs and the evidence for each belief, and then when you get new evidence for a belief, add it to the old evidence and re-evaluate. For example, replacing an old wives' tale with a peer-reviewed study is (usually) a no-brainer. On the other hand, if you have conflicting peer-reviewed studies, then your confidence in both should decrease and you should go back to the old wives' tale (which, being old, is probably useful as a belief, regardless of truth value).

Finally, the defeatist answer is that you can't actually distinguish that you are delusional. With the film Shutter Island in mind, I hope you can see that almost nothing is going to shake delusions; you'll just rationalize them away regardless. If you keep notes on your beliefs, you'll dismiss them as being written by someone else. People will either pander to your fantasy or be dismissed as crooks. Every day will be a new one, starting over from your deluded beliefs. In such a situation there's not much hope for change.

For the record, I disagree with "delusional disorders being quite rare"; I believe D is somewhere between 0.5 and 0.8. Certainly, only 3% of these are "serious", but I could fill a book with all of the ways people believe something that isn't true.