= 27d567bc2d48d9f40e9ccc20ea370b15)
MartinB comments on Scientific Self-Help: The State of Our Knowledge - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (493)
I think you find more people for who it does not work, than for whom it works. But I doubt this works as evidence that it does not work.
There are many factors that influence success. One would be doing it wrong.
Not much, no. But it works as more evidence than people denouncing it. Weak evidence! Weak! Weak.
I think I see different requirements for things working that for things not working.
People getting actively angry about something looks like a (weak) indicator that they are afraid there might be something to it. They do not get angry at 9/11 conspiracists, but laugh about them. They do get angry at the other party.
If you have someone trying to do something and be unsuccessful that tells you even less. I would not see it as indicator of not working. It is just no particular evidence.
But maybe I have the notion of what 'weak' means wrong.
It almost has to be evidence; even if it's just evidence that the person isn't doing it right, then you push this case through your prior for how often they do things wrong. Unless your prior is very high, you're still getting half the impact or more of the evidence. Although you could argue from the selection effect of "correct in your expectations of success" that you're almost guaranteed to only notice cases where they are doing it wrong, not cases where it doesn't work (because you don't expect it to work where it won't).
More evidence than a negative amount? How much is that worth?
For practical purposes, I regard weak evidence as equivalent to no evidence. It has the value of a penny lying on the road: I do not bother to pick it up. Weak evidence is not worth wasting time on.
Further up in the thread, I claimed that (in cases like these) people denouncing it is weak evidence in favour.
Anecdote: I once refused to pick up a small denomination coin (we don't have pennies in Australia anymore). A friend did and promptly discovered it was a penny, from 1938, and worth two dollars to collectors.
On practical matters, though, this is a case where training will greatly decrease effort and time to pick up (more so than for pennies). If you track how much time is spent on picking up pennies and put a value on that amount of time, and then how much money in pennies you have picked up, you will probably find it not worth it. I expect this weak-evidence-picking-up will prove worth it - if it even slightly addresses the common human problem of incorrectly assessing or entirely discounting large amounts of small evidence, it comes out positive for me.
Still not worth it unless you have a personal interest in coins or expect to find or otherwise collect many such coins. (Of course many people would value the experience at more than $2 just for the novelty and the story.)
Can you give some examples of that, i.e. where large amounts of small evidence are being badly used?
This appears to be) an example of an accumulation of weak evidence badly used, but in the opposite direction. (I say "appears", because I haven't read the article it references, just that LW posting and its comments.)
Probabilities of multiple events are being multiplied without concern for whether they are independent. And that is the basic practical problem with accumulating weak evidence. Look at the vast amount of evidence for the existence of Santa Claus! Even if each story only offers a microbit of evidence....well, no. The stories are not independent of each other. Collectively they prove no more than a handful of them do.