You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

HungryHobo comments on What are some science mistakes you made in college? - Less Wrong Discussion

5 Post author: aarongertler 23 March 2014 05:28AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (32)

You are viewing a single comment's thread.

Comment author: HungryHobo 24 March 2014 01:58:40PM 6 points [-]

Not taking account of multiplicity.

Ideally you should plan exactly how you're going to analyse the data before the experiment but in reality students muddle through a bit.

analyzing data in multiple ways is a big no-no if you're just hunting for that elusive 0.05 P value to get it published.

It's stupid and causes statisticians to tear their hair out(both the arbitrary requirement a lot of journals set and the bad stats by researcher) but it's the reality in a lot of research.

Doing that can be compensated for as long as you keep track of what you tried and make that data available.

It's even worse because often people, including experienced professors, delude themselves with bad stats and waste time and money chasing statistical phantoms because they went significance mining.