Are ‘Evidence-based’ policies damaging policymaking?

-7 quantropy 22 August 2013 08:58AM

Both the government and the public should be far more sceptical about policies which are purported to be ‘evidence-based’, argues new research released today by the Institute of Economic Affairs.

In Quack Policy – Abusing Science in the Cause of Paternalism, Jamie Whyte exposes how politicians promote regulations and taxes under the justification of scientific evidence, yet the experts promoting these policies often make basic errors and have little or no grasp of economics.

http://www.iea.org.uk/in-the-media/press-release/%E2%80%98evidence-based%E2%80%99-policies-are-damaging-uk-policymaking

Those in favour of evidenced based policies have tended to be dismissive, arguing that this is just a case of lack of evidence, rather than a problem with evidence-based policies per se.

https://twitter.com/bengoldacre/status/370159847985381376

I'm not fully convinced though.  I've started reading Taking the medicine: a short history of medicine's beautiful idea and our difficulty swallowing it by Druin Burch, and this has plenty of examples of people saying something like "Of course, up till now medicine has been totally wrong, but I've got a new way of doing it right" - and then going on to make the same old mistakes. Maybe evidence based practice is the same, just a way of convincing ourselves that we've got it right this time.

As I see it, the trouble with evidence based practice is that what counts as evidence based today doesn't meet tomorrows higher standards.  This means

  1. How can we trust evidence based practice if it might be overturned tomorrow?
  2. As less gets to count as evidence, evidence based practice gets harder and harder to do.

At the root of this is the insistence that evidence must meet some 'gold standard'.  This seems to be too much the frequentist viewpoint, but in the end we have to be Bayesians, taking all evidence into account.

Comment author: quantropy 19 August 2013 06:14:36AM 3 points [-]

As I see it the question isn't so much about elites and non-elites, but about problem solvers and storytellers. Storytellers give you a reason to want to cross the ocean, problem solvers make sure that you have boat that doesn't sink. Much of the internet is populated by storytellers - someone might do a lot of problem solving in their day job, and then write stories about it in a blog.

MathOverflow, the top site cited as what Less Wrong should be aiming for, is very much about problem solving - you post a problem and can hope to get an answer from some really bright people within a couple of hours. But Jonah Sinick asked why more winners of mathematical prizes don't post their thoughts for all to see - well maybe that isn't the way to win prizes.

It's hard to see where Less Wrong fits into this division. One would think that rationality would be about getting away from the stories that we have been told and getting down to problem solving - new ways to deal with akrasia, poverty or death. Less Wrong looks like it ought to be the public (i.e. storytelling) face of an enterprise which goes in for such problem solving - that is what internet forums often are. But when you dig deeper, you just tend to find more storytelling. I would guess that Elizier has excellent problem solving skills, but the Elizier we see is very much a storyteller.

In the end it's hard to see what Less Wrong is about. Informal discussion is all very well, but to get more interest from elsewhere it needs more of a sense of focus - what sort of problems should we be looking at and what is being done to tackle them.

Comment author: quantropy 16 March 2012 11:34:48AM *  4 points [-]

As I see it Cult =Clique + Weird Ideas

I think the weird ideas are an integral part of LessWrong, and any attempt to disguise them with a fluffy introduction would be counterproductive.

What about Cliquishness? I think that the problem here is that any internet forum tends to become a clique. To take part you need to read through lots of posts, so it requires quite a commitment. Then there is always some indication of your status within the group - Karma score in this case.

My advice would be to link to some non-internet things. Why not have the FHI news feed and links to a few relevant books on Amazon in the column on the right?

Comment author: quantropy 22 February 2012 11:38:13AM 3 points [-]

I'm not convinced that this is either workable or desirable. Not workable because it would soon become unwieldy trying to remember the network of sources from which our beliefs originate. Not desirable because it would lead to us judging the reliability of a belief by looking at the reliability of the sources, rather than by looking for some independent confirmation or refutation. Hence we would be more likely to accept something that 'everybody knows' because more sources seem to lead to higher probability.

Also, what do we do when we find that there isn't a bridge at 234567. Do we go back to each of our sources and try to persuade them that they are wrong (and that they should similarly notify their sources and anyone else that they have passed the information on to). Isn't it better to make a general announcement: Contrary to what is believed there is no bridge at 234567 (and to add the evidence to support this assertion)

Comment author: quantropy 06 November 2011 06:29:01PM 0 points [-]

The problem is that you are betting away the money needed for your living expenses, as well as that which you would put towards savings.

If you go the saving route, then 12 minutes of each hour goes in living expenses, 48 minutes to your savings, which after 5.5 years will total 10000 hours. If you bet the hour in a lottery and lose then you don't have the 12 minutes required for living expenses.

Comment author: rhollerith_dot_com 28 May 2011 06:27:36PM 0 points [-]

Thanks.

I am now curious in what other traits or values correlate with someone's place in this bimodal distribution.

Comment author: quantropy 28 May 2011 08:32:21PM 0 points [-]

I still haven't been able to find any of the early research which I remember, but searching for "standard gamble" on google brings up lots of related results

Comment author: rhollerith_dot_com 28 May 2011 03:58:12PM 1 point [-]

Interesting. (Please do not continue to search for refs, but I'd be interested in any tidbits of how you came to the knowledge, e.g., "I read papers on medical procedures as part of my job".)

I am in the first category: I'd accept a 70% risk of death to have the kind of health I would have had if I'd never contracted Lyme disease.

Comment author: quantropy 28 May 2011 04:53:40PM 0 points [-]

I'm not sure how I came to know this - I think I first heard of it over 30 years ago. I've probably read a number of books about risk since then, some of which may have also mentioned the bimodal distribution, but I'm afraid I can't remember one.

Comment author: quantropy 28 May 2011 12:52:51PM 2 points [-]

The risks people are willing to take to live a 'normal' life have a surprisingly bimodal distribution, with some accepting say 70% risk of death, whereas others only being willing to accept a minimal risk (I've been searching for references to studies showing this, but haven't been able to find any via Google). I think that lesswrongers are likely to fall into the second category, especially in a case like this where dialysis/transplantation may be tedious, but they don't exclude the possibility of undergoing the stem cell treatment in a few years time, when the risks have been ironed out.

In response to lessannoying.org
Comment author: quantropy 28 May 2011 08:45:03AM 4 points [-]

It means two things: 1: The author, Kenneth Myers - a lesswronger himself apparently - wants to give advice on socializing to the rest of the community. I'm not sure how good the advice is.

2: He sees something odd about lesswrong, which isn't just that lesswrongers have different beliefs to other people. lesswrong is not just a bunch of people discussing weird ideas (in which case it would be ignorable), but neither is it discussing ideas which are part of a traditional academic discipline (in which case it would also be ignorable, as it would be the published work that would be important). I think that it's this in-betweenness which makes people uneasy

Comment author: quantropy 17 May 2011 08:57:12AM 1 point [-]

It has been suggested that interesting things, in particular life, happen where there are entropy gradients (I'm thinking of 'Into the Cool' by Eric Schneider and Dorion Sagan). If Ramsey theory could be used to show that most universes are likely to have entropy gradients then that could be used to argue that fine tuning is unnecessary.

The fact that this would only be a tiny part of the universe agrees with the observation that the only life we know about occupies a tiny part of the universe.

Regarding Conway's game of life, it's important to note that it allows irreversible microphysics, and so won't have anything like our thermodynamics.

View more: Next