Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

A Federal Judge on Biases in the Criminal Justice System.

22 Costanza 03 July 2015 03:17AM

A well-known American federal appellate judge, Alex Kozinski, has written a commentary on systemic biases and institutional myths in the criminal justice system.

The basic thrust of his criticism will be familiar to readers of the sequences and rationalists generally. Lots about cognitive biases (but some specific criticisms of fingerprints and DNA evidence as well). Still, it's interesting that a prominent federal judge -- the youngest when appointed, and later chief of the Ninth Circuit -- would treat some sacred cows of the judiciary so ruthlessly. 

This is specifically a criticism of U.S. criminal justice, but, ceteris paribus, much of it applies not only to other areas of U.S. law, but to legal practices throughout the world as well.

Comment author: KnaveOfAllTrades 20 July 2014 04:00:18PM 3 points [-]

There might be a trainable skill of noticing high-leverage facts. For example, a friend of mine claims that a modest weekly investment in keeping up with soccer allows him to 'have a good conversation with any man'. In particular example, a basic knowledge of the proceedings of a FIFA World Cup while it is taking place is extremely cheap but might be extremely high leverage depending on where one lives and the demographic distribution of one's interlocutors.

Similarly 'current affairs' (legislation, policy, political manoeuvrings, disasters, humorous stories, etc.)

Also, I encourage this kind of experimentation post.

Comment author: Costanza 20 July 2014 07:51:44PM 1 point [-]

There's an app for that, at least on the IT Crowd.

Comment author: James_Miller 18 July 2014 05:52:26AM *  29 points [-]

Since LW is going to get a lot of visitors someone should put an old post that would make an excellent first impression in a prominent position. I nominate How to Be Happy.

Comment author: Costanza 18 July 2014 10:25:07PM *  2 points [-]

Let's check featured articles on the main page on 19 July 2014....and...there we go.

Comment author: jsteinhardt 13 July 2014 11:29:38PM *  7 points [-]

Rather, the problem is that at least one celebrated authority in the field hates that, and would prefer much, much more deference to authority.

I don't think this is true at all. His points against replicability are very valid and match my experience as a researcher. In particular:

Because experiments can be undermined by a vast number of practical mistakes, the likeliest explanation for any failed replication will always be that the replicator bungled something along the way.

This is a very real issue and I think that if we want to solve the current issues with science we need to be honest about this, rather than close our eyes and repeat the mantra that replication will solve everything. And it's not like he's arguing against accountability. Even in your quoted passage he says:

The field of social psychology can be improved, but not by the publication of negative findings. Experimenters should be encouraged to restrict their “degrees of freedom,” for example, by specifying designs in advance.

Now, I think he goes too far by saying that no negative findings should be published; but I think they need to be held to a high standard for the very reason he gives. On the other hand, positive findings should also be held to a higher standard.

Note that there are people much wiser than me (such as Andrew Gelman) who disagree with me; Gelman is dissatisfied with the current presumption that published research is correct. I certainly agree with this but for the same reasons that Mitchell gives, I don't think that merely publishing negative results can fix this issue.

Either way, I think you are being quite uncharitable to Mitchell.

Comment author: Costanza 14 July 2014 12:14:35AM 38 points [-]

What is the purpose of an experiment in science? For instance, in the field of social psychology? For instance,what is the current value of the Milgram experiment? A few people in Connecticut did something in a room at Yale in 1961. Who cares? Maybe it's just gossip from half a century ago.

However, some people would have us believe that this experiment has broader significance, beyond the strict parameters of the original experiment, and has implications for (for example) the military in Texas and corporations in California.

Maybe these people are wrong. Maybe the Milgram experiment was a one-off fluke. If so, then let's stop mentioning it in every intro to psych textbook. While we're at it, why the hell was that experiment funded, anyway? Why should we bother funding any further social psychology experiments?

I would have thought, though, that most social psychologists would believe that the Milgram experiment has predictive significance for the real world. A Bayesian who knows about the results of the Milgram experiment should better be able to anticipate what happens in the real world. This is what an experiment is for. It changes your expectations.

However, if a supposedly scientific experiment does nothing at all to alter your expectations, it has told you nothing. You are just as ignorant as you were before the experiment. It was a waste.

Social psychology purports to predict what will happen in the real world. This is what would qualify it as a science. Jason Mitchell is saying it cannot even predict what will happen in a replicated experiment. In so doing, he is proclaiming to the world that he personally has learned nothing from the experiments of social psychology. He is ignorant of what will happen if the experiment is replicated. I am not being uncharitable to Mitchell. He is rejecting the foundations of his own field. He is not a scientist.

Comment author: James_Miller 13 July 2014 09:30:04PM 7 points [-]

When natural scientists attempt to replicate famous experiments where the original result was clearly correct, with what probability do they tend to succeed? Is it closer to 1 than, say, .7?

Comment author: Costanza 13 July 2014 09:51:40PM 11 points [-]

I'd think that "famous experiments where the original result was clearly correct" are exactly those whose results have already been replicated repeatedly. If they haven't been replicated they may well be famous -- Stanford prison experiment, I'm looking at you -- but they aren't clearly correct.

This is why we can't have social science

36 Costanza 13 July 2014 09:04PM

Jason Mitchell is [edit: has been] the John L. Loeb Associate Professor of the Social Sciences at Harvard. He has won the National Academy of Science's Troland Award as well as the Association for Psychological Science's Janet Taylor Spence Award for Transformative Early Career Contribution.

Here, he argues against the principle of replicability of experiments in science. Apparently, it's disrespectful, and presumptively wrong.

Recent hand-wringing over failed replications in social psychology is largely pointless, because unsuccessful experiments have no meaningful scientific value.

Because experiments can be undermined by a vast number of practical mistakes, the likeliest explanation for any failed replication will always be that the replicator bungled something along the way. Unless direct replications are conducted by flawless experimenters, nothing interesting can be learned from them.

Three standard rejoinders to this critique are considered and rejected. Despite claims to the contrary, failed replications do not provide meaningful information if they closely follow original methodology; they do not necessarily identify effects that may be too small or flimsy to be worth studying; and they cannot contribute to a cumulative understanding of scientific phenomena.

Replication efforts appear to reflect strong prior expectations that published findings are not reliable, and as such, do not constitute scientific output.

The field of social psychology can be improved, but not by the publication of negative findings. Experimenters should be encouraged to restrict their “degrees of freedom,” for example, by specifying designs in advance.

Whether they mean to or not, authors and editors of failed replications are publicly impugning the scientific integrity of their colleagues. Targets of failed replications are justifiably upset, particularly given the inadequate basis for replicators’ extraordinary claims.

This is why we can't have social science. Not because the subject is not amenable to the scientific method -- it obviously is. People are conducting controlled experiments and other people are attempting to replicate the results. So far, so good. Rather, the problem is that at least one celebrated authority in the field hates that, and would prefer much, much more deference to authority.

In response to comment by [deleted] on Less Wrong’s political bias
Comment author: Sophronius 25 October 2013 06:41:26PM *  10 points [-]

Ok, here are my reasons:

1) I would like to be able to talk about politics with rational people
2) Understanding more of how the world works could be useful in other areas.
3) I want to be able to make references to things that might be construed as political without having the entire post downvoted to -6 because I'm not allowed to talk about politics.
4) I am increasingly worried about the radicalisation (Assuming it really is increasing) of Less Wrong and I think the problem is that crazy views get too much credence here, due to an unwillingness to criticize by more rational people. (Biggest issue for me)

Edit: I don't get why I receive so many downvotes in a matter of minutes for answering a question as honestly and helpfully as I can manage. I see the same in some of my other posts. I somewhat suspect this is entirely based on party politics, where I am perceived to be criticizing party X in the original post, and so have unrelated posts downvoted by angry people. But maybe I'm missing something.

Comment author: Costanza 25 October 2013 09:10:52PM *  5 points [-]

I would like to be able to talk about politics with rational people ...[but]...the problem is that crazy views get too much credence here, due to an unwillingness to criticize by more rational people.

Right. It's those damn greens. Damn those greens, with their votes for... crazy green things! Not like us blues, who want nothing but good and rational blueness!

[ETA] My mind has been killed. This is why I don't want party politics -- as opposed to policy -- on LessWrong.

In response to comment by [deleted] on Less Wrong’s political bias
Comment author: Sophronius 25 October 2013 06:41:26PM *  10 points [-]

Ok, here are my reasons:

1) I would like to be able to talk about politics with rational people
2) Understanding more of how the world works could be useful in other areas.
3) I want to be able to make references to things that might be construed as political without having the entire post downvoted to -6 because I'm not allowed to talk about politics.
4) I am increasingly worried about the radicalisation (Assuming it really is increasing) of Less Wrong and I think the problem is that crazy views get too much credence here, due to an unwillingness to criticize by more rational people. (Biggest issue for me)

Edit: I don't get why I receive so many downvotes in a matter of minutes for answering a question as honestly and helpfully as I can manage. I see the same in some of my other posts. I somewhat suspect this is entirely based on party politics, where I am perceived to be criticizing party X in the original post, and so have unrelated posts downvoted by angry people. But maybe I'm missing something.

Comment author: Costanza 25 October 2013 07:29:42PM 6 points [-]

1) I would like to be able to talk about politics with rational people

I'd suggest a distinction between "politics" and "policy", at least in the American English prevalent on LessWrong. "Politics" implies party politics, blue versus green, horse races (by which I mean election horse races), and tribalism. I think your post suggested an interest in this. Personally, I don't want this here.

If, however, you want to talk about policy, using the analytical language of policy, then I say go for it. However, your original post, with its reference to parties, made me doubtful.

In response to comment by [deleted] on Less Wrong’s political bias
Comment author: Sophronius 25 October 2013 06:51:34PM 2 points [-]

I think it's a bit silly to call it "courageous" to criticize an online forum. At worst it makes me feel slightly bad when my posts get downvoted as a result. But I appreciate that you are trying to encourage meaningful criticism on Less Wrong, which I feel is badly needed. So thank you for that.

Upvoted for trying to make Less Wrong a better place.

Comment author: Costanza 25 October 2013 06:58:34PM 1 point [-]

I think it's a bit silly to call it "courageous" to criticize an online forum. At worst it makes me feel slightly bad when my posts get downvoted as a result.

Well said! Well said indeed! And for that I will award you...a karma point!

Comment author: [deleted] 25 October 2013 06:03:02PM 0 points [-]

Upvoted because even if the answer is "no," it's still a question worth asking and one that takes courage to ask.

In response to comment by [deleted] on Less Wrong’s political bias
Comment author: Costanza 25 October 2013 06:31:22PM 7 points [-]

Downvoted because the original post didn't so much ask a question as make an assertion which I personally didn't find so valuable. As you point out, why would anyone come here for political discussion in the first place? So I downvoted it, because that's what the karma system is for. In the end, a karma point is just a karma point. Nothing personal in it.

View more: Next