Somehow I couldn't quite figure out what Eliezer was advocating when I first read this article. Now I think that he wants to see more exchanges like the following:

Experimentalist: My experiment yielded results that contradict your theory at the p < 0.01 level!

Theorist: I remain unconvinced. I defy your data.

Experimentalist: What?! You can't just ignore empirical observations like that. So, which is it? Are you accusing me of negligence or fraud?

Theorist: I'm not accusing you of anything. But your experiment could have been one of the 1-in-100 that would get results at least that strong just by chance. As unlikely as it is that you were so unlucky, it is nonetheless more likely than that my theory is wrong. Your p-value just wasn't large enough to kill my theory in one fell swoop. However, several independent replications of your results would be enough to do it.