Desrtopa comments on Experiment Idea Thread - Spring 2011 - Less Wrong

28 Post author: Psychohistorian 06 May 2011 06:10PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (53)

You are viewing a single comment's thread. Show more comments above.

Comment author: Louie 09 May 2011 09:49:18PM 0 points [-]

the right response is to try to do better

Agreed. Better is better.

and nail down our results with more precision, not less

I disagree. For example, you can rule out Dual N-Back as a possible Intelligence Amplification intervention with less precision than Jaeggi used to repeatedly mis-prove it as one. Depends on what you mean by precision I suppose. If you mean more time, effort, people, and statistical significance then precision is not needed. If by precision, you just mean being more right... well, I agree, we should be more right.

Most bogus science is very precise: That's why it looks stronger than it is. Poor methodology and experimental design will still allow someone to prove any correlation with p < 0.05 significance. If I want to disprove someone who published an incorrect result, should I have to expend more time, people, and resources than they used just to over-prove the counter-claim with "more precision" -- even though their claim was never wrong due to "lack of precision" in the first place?

Calling for "more precision" is like calling for "more preparation". It has 100% applause appeal and costs nothing for people to call for. But it costs people actually doing research a lot of time. When you advocate for smarter people to use "more precision", you're also advocating for smarter people to do "less research"... the extra precision comes from somewhere.

Are you actually in favor of smarter people doing less research than they currently do?

Comment author: Desrtopa 11 May 2011 04:10:21AM 2 points [-]

Are you actually in favor of smarter people doing less research than they currently do?

Considering how much research, given the low levels of confidence warranted by its methodology, is essentially worthless, yes, I am willing to say without reservation that there is dead wood to cut away.