= 27d567bc2d48d9f40e9ccc20ea370b15)
Azathoth123 comments on This is why we can't have social science - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (82)
The amusing thing is that Mitchel's argument proves much more than he wants it to prove.
Notice that the above argument applies just as well to the original experiment being replicated.
Yes, noticed.
Has anyone read his entire article? Does he attempt any justification for why this particular argument doesn't equally apply to the original experiment?
One principle I try to keep in mind is "The other guy is probably not a total moron. If it seems that way, you're probably missing something."
I read it. He has a section titled "The asymmetry between positive and negative evidence".
His argument is that a positive result is like seeing a black swan, and a null result is like seeing a white swan, and once you see a black swan, then no matter how many white swans you see it doesn't prove that all swans are white.
He addresses the objection that this leaves us unable to ever reject a spurious claim. His answer is that, since negative evidence is always meaningless, we should get positive evidence that the experimenter was wrong.
I think this is a fair summary of the section. It's not long, so you can check for yourself. I am... not impressed.
Actually, it's like hearing a report of a black swan, which is why the burden of proof is generally put on the report.
It's even worse than that for him. What a bad analogy for him to rest his case on. Surely, the purpose of these social science studies is not to make a claim about the existence of some bizarre subset of the population (a black swan), but that the results will generalize to the population at large (all swans are black).
That's more than enough for me.
Thanks for taking the bullet for us.
There's a lot wrong with the argument; he has no actual justification for assuming that social science is anything like swan-spotting.
But even within his unjustified analogy... apparently if someone reports a new color of swan in Australia, he might give polygraphs and vision tests to the reporter, but sending an expedition to Australia to check it out would be of no scientific value.
Medicine has had its Ioannidis. Is anyone doing the same for psychology and social science?
Uri Simonsohn?
There's El Vul's Voodoo Correlations in Social Neuroscience.
My interpretation is that he's saying false positives are unlikely but false negatives are common. If that's the case, then getting a positive result should greatly affect your beliefs; but a negative replication shouldn't shift them much.
(I can't actually justify this interpretation by reading the text, but it does make the text seem fairly reasonable.)
Beautiful point.
Congratulations, you're smarter than a Harvard professor. I noticed the same thing, so I'm also smarter than a Harvard professor. I would hope that everyone on this site is smarter than a Harvard professor. A Harvard professor who got his B.A. and M.S. from Yale and his Ph.D. from Harvard.
This is sociologically interesting: is it an isolated incident that a Harvard professor -- an award-winning one, no less -- would so loudly fail to comprehend how science and statistics work? That doesn't seem likely, for two reasons. First, he probably talked it over with other people in the field, or at least mentioned why he thought replication was bad and wrong, and no one talked him out of it. Second, the system he came through produced him, and it's unlikely that that sort of error would only be produced once, since education is fairly standardized. So: what does this say about the relevant institutions? (The social sciences, HYP, academia, etc.)
I don't think that means you are smarter than that Harvard professor. He is a very successful person and has reached heights coveted by many very smart people. It just means that the game he is playing is not one where you get ahead by saying things that make sense.
For example, if you listen to a successful politician and spot a false statement he utters, that does not mean that you are smarter than that politician.
Yes, academics are supposed to raise the status of their institution. This brings in money, which helps educate students, which makes the world a better place. Unsuccessful replication threatens this. Plus, replication does nothing to advance social justice.
His [website (http://www.wjh.harvard.edu/~scanlab/people.html) describes his research as:
That doesn't seem to me like someone who focus on advancing social justice.
You should have looked at his vita for a more accurate description of his activities. If you had looked at his paper titles, some of them indicate he's not a stranger to social justice like theorizing and investigation, and likewise his funding sources, on top of Harvard's well-earned reputation: eg. "What’s in a forename?: Cue familiarity and stereotypical thinking", "Gender differences in implicit weight identity", "Deflecting negative self-relevant stereotype activation: The effects of individuation", "Me and my group: Cultural status can disrupt cognitive consistency", and the funding:
Sorry my mistake. As far as the paper titles goes.
As far as the funding goes the National Science Foundation isn't an entity that I would see as spearheading the social justice movement.
The same goes for the Templeton Foundation. They have the reputation of wanting "progress in spiritual discoveries" instead of "advancing social justice".
Both groups are quite big and may fund more than you think; not that the grantors always get what they think they're getting or are the only people who are then allowed to draw upon the research. For example, consider "stereotype threat", much beloved of social-justice types for explaining how bad white people keep test scores low for women and blacks; you can see the NSF certainly has been involved in that research in the past just with a cursory google: http://scholar.google.com/scholar?q=%22stereotype%20threat%22%20%22National%20Science%20Foundation%22 (I count ~4 NSF grants attested to just from the snippet-view for the first page).
I do accept that both groups do fund a large variety of courses but I still wouldn't conclude from the funding source that there a bias in the direction of the social justice movement.
When thinking about the title "The neural basis of stereotyping" you might be right that it smells like pseudoscience. It's a bit like the "The neural basis of acupuncture". Instead of searching for the neural basis it would make much more sense to focus of studying the actual effect.
I wonder if I picked up to much stereotypes about fMRI research ;)
I never said the NSF is biased in favor of SJ. I said if you had bothered to look at the vita instead of stopping at the most convenient place, you would have found a number of paper and grant titles which indicate a more than theoretical interest in topics strongly associated with SJ on top of his affiliation with an institute with a strong background both current and historical in liberal thought & SJ-like figures such as Cornell West and to borrow from parallel ongoing conversations, crucified Sumners for his well-founded suggestion that the elite math achievement gap might be a necessary consequence of gender differences. And then you said the NSF was evidence against SJ association, which is either wrong or weak evidence since they fund related research all the time.
He teaches at an elite university so it's highly probably he greatly values social justice.
Either you don't read your own comments, or you classify all social justice as signaling.
In the latter case, I'm sure you'll come up with reasons why you aren't making a fully general counterargument. I would urge you to stop for a second and ask concretely what you would have thought about, say, abolitionists.
I don't understand what you mean. Social justice is a label for a set of beliefs. I do not think that social justice is just about signaling. If it helps to understand what I mean, I'm a politically incorrect college professor.
Are you implying that there is a contradiction in what you quoted? If so, I don't see it.