James_K comments on Open Thread: June 2010 - Less Wrong

5 Post author: Morendil 01 June 2010 06:04PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (651)

You are viewing a single comment's thread.

Comment author: James_K 01 June 2010 08:46:54PM 8 points [-]

This post is about the distinctions between Traditional and Bayesian Rationality, specifically the difference between refusing to hold a position on an idea until a burden of proof is met versus Bayesian updating.

Good quality government policy is an important issue to me (it's my Something to Protect, or the closest I have to one), and I tend to approach rationality from that perspective. This gives me a different perspective from many of my fellow aspiring rationalists here at Less Wrong.

There are two major epistemological challenges in policy advice, in addition to the normal difficulties we all have to deal with: 1) Policy questions fall almost entirely within the social sciences. That means the quality of evidence is much lower than it is in the physical sciences. Uncontrolled observations, analysed with statistical techniques, are generally the strongest possible evidence, and sometimes you have nothing but theory or professional instinct to work with.
2) You have a very limited time in which to find an answer. Cabinet Ministers often want an answer within weeks, a timeframe measured in months is luxurious. And often a policy proposal is too sensitive to discuss with the general public, or sometimes with anyone outside your team.

By the standards of Traditional Rationality, policy advice is often made without meeting a burden of proof. Best guesses and theoretical considerations are too weak to reach conclusions. A proper practitioner of Traditional Rationality wouldn't be able to make any kind of recommendation, one could identify some promising initial hypotheses, but that's it.

But Just because you didn't have time to come up with a good answer doesn't mean that Ministers don't expect an answer. And a practitioner of Bayesian Rationality always has a best guess as to what is true, even if the evidence base is non-existent you can fall back on your prior. You don't want to be overconfident in stating your position, assumptions must be outlined and sensitivities should be explored. But you still need to give an answer and that's what attracts me to Bayesian approaches: you don't have to be officially agnostic until being presented with a level of evidence that is unrealistically high for policy work.

It seems to me that if you have very good quality evidence then Bayesian and Traditional Rationality are very similar. Good evidence either proves or disproves a proposition for a Traditional Rationalist, and for a Bayesian Rationalist it will shift their probability estimate, as well as increasing their confidence a lot. The biggest difference seem to me to be that Bayesian Rationality seems is able to make use of weak evidence in a way Traditional Rationality can't.

Comment author: xamdam 02 June 2010 04:15:26PM 1 point [-]

Reminded me of one of my favorite movie dialogues - from Sunshine. Context was actually physics, but the complexity of the situation and the time frame but the characters in the same situation as you with the Cabinet ministers.

Capa: It's the problem right there. Between the boosters and the gravity of the sun the velocity of the payload will get so great that space and time will become smeared together and everything will distort. Everything will be unquantifiable.

Kaneda: You have to come down on one side or the other. I need a decision.

Capa: It's not a decision, it's a guess. It's like flipping a coin and asking me to decide whether it will be heads or tails.

Kaneda: And?

Capa: Heads... We harvested all Earth's resources to make this payload. This is humanity's last chance... our last, best chance... Searle's argument is sound. Two last chances are better than one.

http://www.imdb.com/title/tt0448134/quotes?qt0386955

Comment author: James_K 02 June 2010 10:06:19PM 1 point [-]

Yes, that's a good example. There are times when a decision has to be made, and saying you don't know isn't very useful. Even if you have very little to go on, you still have to decide one way or the other.

Comment author: realitygrill 02 June 2010 03:05:53AM 0 points [-]

I am not at all like you. I don't have much interest in policy at all, and I do tend to refuse to hold a position, being very mindful of how easy it is to be completely off course (Probably from reading too much history of science. It's "the graveyard of dead ideas", after all.). I'm likely to tell the Cabinet Ministers to get off my back or they'll have absolutely useless recommendations.

However, I think you have hit upon the point that makes Bayesianism attractive to me: it's rationality you can use to act in real-time, under uncertainty, in normal life. Traditional Rationality is slow.

Comment author: James_K 02 June 2010 03:38:38AM 0 points [-]

I see your point, the trouble is that a recommendation that comes too late often is absolutely useless. A lot of policy is time-dependant, if you don't act within a certain time frame then you might a swell do nothing. While sometimes doing nothing is the right thing to do, a late recommendation is often no better than no recommendation.

Comment author: realitygrill 02 June 2010 05:02:12AM 0 points [-]

Yeah, I forgot to add that you've budged me slightly from my staunch positivist attitude for social science. Thanks. Reading up on complex adaptive systems has made me just that much more skeptical about our ability to predict policy's effects, and perhaps biased me.

Comment author: James_K 02 June 2010 06:05:33AM 1 point [-]

It's nice to know I've had an influence :)

As it happens, I'm pretty sceptical as to how much we can know as well. There's nothing like doing policy to gain an understanding of how messy it can be. While the social sciences have a less than wonderful record in developing knowledge (look at the record of development economics, as one example), and economic forecasting is still not much better than voodoo but it's not like there's another group out there with all the answers. We don't have all of the answers, or even most of them, but we're better than nothing, which is the only alternative.

Comment author: matt 02 June 2010 09:48:18PM *  5 points [-]

Nothing is often a pretty good alternative. Government action always comes at a cost, even if only the deadweight loss of taxation (keyphrase "public choice" for reasons you might expect the cost to be higher than that). I'm not trying to turn this into a political debate, but you should consider doing nothing not necessarily a bad thing, and what you do not necessarily better.

Comment author: James_K 03 June 2010 05:48:22AM 2 points [-]

When I said "better than nothing" I was referring to advice, not the actual actions taken. My background is in economics so I'm quite familiar with both dead-weight loss of taxation and public choice theory, though these days I lean more toward Bryan Caplan's rational irrationality theory of government failure.

I agree that nothing is often a good thing for governments to do, and in many cases that is the advice that Cabinet receives.

Comment author: mattnewport 02 June 2010 10:01:50PM 1 point [-]

Politicians' logic: “Something must be done. This is something. Therefore we must do it.”