MichaelBishop comments on Rationality Quotes - June 2009 - Less Wrong

8 Post author: pjeby 14 June 2009 10:00PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (168)

You are viewing a single comment's thread. Show more comments above.

Comment author: pjeby 17 June 2009 07:27:36PM 1 point [-]

Tell that to a marketing agency.

Actually, marketers are well aware that statistics don't tell them HOW people work; they only tell them what gets the most response. Knowing that a "Johnson box" imrpoves results on mailing A to list B only suggests that it might work with mailing C to list D; it does not tell you how or why it worked, nor does it give you any real way to explain the result when it doesn't work.

Most marketing is useful folklore; good theories in marketing are very few and far between. Even the best teachers of marketing rarely rise to strong theories; the ones that do are mostly either borrowing their models from NLP and hypnosis, or reinventing them.

The quote is not about whether statistics might tell you something useful about people in general, it's about understanding HOW a specific individual is doing what they're doing. A statistical model can suggest paths to investigate, but it can't tell you what's actually going on in a specific case.

IOW, it's an epic FAIL for a mechanic to "rely on statistics as a way of understanding how" your car works (or doesn't) instead of actually observing what a specific car is (or is not) doing.

Comment author: MichaelBishop 17 June 2009 08:28:21PM 2 points [-]

A good mechanic will often use the following reason: 90% of cars with symptom x problem Y, so that is what I will check first.

Comment author: RichardKennaway 18 June 2009 09:00:44AM *  1 point [-]

A good mechanic will often use the following reason: 90% of cars with symptom x problem Y, so that is what I will check first.

Then, he will discover whether Y is actually the problem (ETA: for this particular car), and if not, discard that hypothesis and look for something else. This essential step is missing from all papers in psychology reporting statistical results. The fault is even worse when those results are reported in terms such as (to take a recent example) "willpower is a scarce resource".