LESSWRONG
LW

Comment Permalink
johnlawrenceaspden00

Well, I don't think the big diagram is 'a good idea'. 'Making a big diagram' is a good idea. And all the techniques used to make it are good ideas. Similarly the map of England is not simple, but the idea of map-making and the techniques of surveying are.

As far as purpose goes, I think you're right.

We can ask, in evolutionary mode, "what if anything gave this a selective advantage?" or, relatedly, "why didn't this costly thing get selected out of existence?". And we can ask "what does this actually do?". What does talk of > purpose add beyond these?

If it were the case that talk of purpose couldn't be reduced to those things, and it were actually implying a fully intelligent mind, or some sort of intelligence to evolution beyond 'optimization process proceeding by local hill-climbing', then I'd get very suspicious indeed.

However, I just like teleological thinking. It's a bit like infinitesimals in analysis. They're not a bad way to get out quick and dirty results, but they can mislead. The careful man goes back and does the epsilon-delta thing as well. But I don't like to discard sources of intuition. We have so few!

And I think that a lot of the reason for mistrust of purpose is historical silliness like 'We die so that there will be room for the young'. That doesn't work at all for a sexual species. But that sort of thinking might be helpful for something undergoing group-selection. And pathogens, I think, often reproduce by cloning. One brave virus heading off to take out the bridge might well be genetically fulfilled by its sisters busy looting the cargo hold.

Reply
gjm10

I don't think the big diagram is 'a good idea'. 'Making a big diagram' is a good idea.

That's a very reasonable distinction. But then -- bringing the analogy back to the motivating example of thyroid-y things -- why not say 'these symptoms are caused by deficiency of, or insensitivity to, thyroid hormones' and 'these symptoms are caused by [INSANELY COMPLEX EXPLANATION WE DON'T ACTUALLY HAVE GOES HERE]' are not 'good ideas' -- the good idea is something like 'figuring out what causes the symptoms'?

I mean, isn't that pretty much the level of abstraction t... (read more)

Reply
See in context
by johnlawrenceaspden
3 min read
53

16

I've been thinking, and writing, about The Impossible Question of the Thyroid for some while now.

I came up with what I thought was a good stab at an answer to its majestic mystery:

http://lesswrong.com/r/discussion/lw/nef/the_thyroid_madness_core_argument_evidence/

This is a very simple and obvious explanation of an awful lot of otherwise confusing data, anecdotes, quackery, expert opinion and medical research.

People seem to hate it because it is so simple, and makes so many predictions, most of which are terrifying.

And it is obviously false! Of course medicine has tried using thyroid supplementation to fix 'tired all the time'. It doesn't work!

EDIT: Apparently I spoke too soon. GRB Skinner tried it in 2000, and it works a treat. See comments.

But there really is an awful lot unexplained about all this T4/T3 business, and why different people think it works differently. I refer you to the internet for all the unexplained things.

In just the endocrinological literature there is a long fight going on about T4/T3 ratios in thyroid supplementation, and about the question of whether or not to treat 'subclinical hypothyroidism'. Some people show symptoms with very low TSH values. Some people have extremely high TSH values and show no symptoms at all.

I've been trying various ways of explaining it all for nearly four months now. And I've found lots of magical thinking in conventional medicine, and lots of waving away of the reports of honest-sounding empiricists, real doctors, who have made no obvious errors of reasoning, most of whom are taking terrible risks with their own careers in order to, as they see it, help their patients.

I've read lots of people saying 'we tried this, and it works', and no people saying 'we tried this, and it makes no difference'. The explanation favoured by conventional medicine strongly predicts 'we tried this, and it makes no difference'. But they've never tried it!

It's really confusing. A lot of people are very confused.

I think that simple explanations are extra-worth looking at because they are simple.

Of course that doesn't mean they're right. Consequences and experiment are the only judge of that.

I do not think I am right! There is no way I can have got the whole picture. I can't explain, for instance: 'euthyroid sick syndrome'. But I don't predict that it doesn't exist either.

But you should look very carefully at the simple beautiful ideas that seem to explain everything, but that look untrue.

Firstly because Solomonoff induction looks like a good way to think about the world. Or call it Occam's Razor if you prefer. It is straightforward Bayesianism, as David Mackay points out in Information Theory, Inference, and Learning Algorithms.

Secondly because all the good ideas have turned out to be simple, and could have been spotted, (and often were) by the Ancient Greeks, and could have been demonstrated by them, if only they'd really thought about it.

Thirdly because experiments not done with the hypothesis in mind have likely neglected important aspects of the problem. (In this case T3 homeostasis, and possible peripheral resistance, and the difference between basal metabolic rate and waking rate, and the difference between core and peripheral temperature, and the possibility of a common DIO2 mutation causing people's systems to react differently to T4 monotherapy, and in general the hideous complexity of the thyroid system and its function in vertebrates in general).

Fourthly because the reason for the 'unreasonable effectiveness of mathematics' is that the simplest ideas tend to come up everywhere!

And so when a mathematician plays with a toy problem for fun, and reasons carefully about it, two thousand years later it can end up winning a major war in a way no one ever expected. 

So that even if there are things you can't explain (I can't explain hot daytime fibro-turks...), you should keep plugging away, to see if you can explain them, if you think hard enough.

Good ideas should be given extra-benefit of the doubt. Not ignored because they prove (slightly) too much!

Do not believe them. Do not ever ever believe them. You will end up worse than Hitler. You will end up worse than Marx.

But give them the benefit of the doubt. Keep them in mind. Try safe experiments, ready to abort when they go wrong.

And if they're easy to refute (mine is), then if you're going to call yourself a scientist, damned well take the trouble to refute the things. You might learn something!

Personal Blog

16

In Defence of Simple Ideas That Explain Everything But Are Wrong
New Comment
53 comments, sorted by
top scoring
Click to highlight new comments since:
[-]gjm90

all the good ideas have turned out to be simple

Yeah, right.

[EDITED to add:] I don't disagree with the overall thrust of the post, and I agree that a lot of good scientific ideas have turned out to be simple. But that was one step too much hyperbole for me.

Reply
[-]roystgnr50

I'd agree that most of the best scientific ideas have been relatively simple... but that's at least partly selection bias.

Compare two possible ideas:

"People with tuberculosis should be given penicillum extract"

"People with tuberculosis should be given (2S,5R,6R)-3,3-dimethyl-7-oxo-6-(2-phenylacetamido)-4-thia-1-azabicyclo[3.2.0]heptane-2-carboxylic acid"

The first idea is no better than the second. But we'd have taken forever to come up with the second, complex idea by just sifting through all the equally-chemically-complex alternatives; we actually came up with it as a refinement of the first, much simpler (in the context of our world) idea. There are surely many even-better complex ideas out there, but searching through idea space brings you to simpler ideas earlier and so they're disproportionately represented.

Reply
[-]johnlawrenceaspden00

And possibly the simple ideas which look true are the shadows of the more complicated truth.

And possibly they are the only path to the truths which we can find.

The puzzle in science is not really 'the unreasonable effectiveness of mathematics'.

It is 'the unreasonable effectiveness of simple ideas'.

And the puzzle, as you say, is that there have always been simple ideas to lead us to the more complicated ideas.

But as you point out, that may well have a simple explanation.

Reply
[-]Gunnar_Zarncke30

That poster is awesome. I'll order one (apparently they are free).

Reply
[-]Dustin10

I can't seem to find a link to order one, where is it?

Reply
[-]gjm30

Here, I think.

Reply
[-]johnlawrenceaspden00

That's complicated g, but the diagram is like a program written in a simple language, and the simple language is atom, element, formula, reaction, enzyme.

The big diagram is contingent, an accident of nature. The underlying ideas, and the techniques used, are simple enough that they can enchant a curious child. As I imagine every reader of this post will well remember.

There are some other simple ideas there too, that have not, I think, been paid as much attention as they deserve in medicine.

Evolution, the war of all against all, predation, parasitism, camouflage, security, identification, group selection, scorched earth defence, stability, trade-off.

And most neglected of all: Purpose.

The iodothyronines are on that diagram. What is their function? What are they for? What were they for? How did they start out? How has their original purpose been modified, step by step, into what it is today?

Reply
[-]gjm00

The big diagram is contingent

Yup. Is that supposed to make it not a counterexample, and if so why? (Note that, e.g., the processes affecting mood, tiredness, etc., are also contingent. You may wish to avoid stipulations that make my counterexample not a counterexample if they also make your leading example not an example :-).)

Again, I'm not disagreeing that many good ideas are simple and that simple ideas are worth pursuing even if you expect that they're never going to be more than useful approximations that may point in helpful directions.

And most neglected of all: Purpose.

My feeling is that if "purpose" is neglected in science it's because it's generally been found to be more misleading than helpful. We can ask, in evolutionary mode, "what if anything gave this a selective advantage?" or, relatedly, "why didn't this costly thing get selected out of existence?". And we can ask "what does this actually do?". What does talk of purpose add beyond these?

It adds something in cases where some actually purposeful agent is responsible for whatever-it-is. So, e.g., I expect it's useful from time to time in finance where the answer to "why do these prices move in this way?" may be "because the owners of these pension funds have these incentives and are acting accordingly", and it's certainly useful in politics or history. But in biology? It seems to me that if you find cases where the full-blown concept of purpose is genuinely better than the alternatives, you've found good evidence[1] for creationism, and so far alleged cases of good evidence for creationism have tended to evaporate on closer inspection.

[1] Of course good evidence is not necessarily anything like proof; sometimes there is good evidence for false things.

Reply
[-]dlarge10

Maybe this is getting too far afield, but I would say that "Purpose" is not only a useful, but an essential heuristic in science when it's being practiced by a kind of entity (like human beings) who are hard-wired to think in terms of purposeful action. Making the first question "What is this for?" brings to bear the full power of uncounted generations of field-tested behaviors, rules of thumb, and search strategies.

It is awfully important, though, not to make it the last question. I guess that's where I'd say yes, a "full-blown concept of purpose" in the sense of an unexplained explanation, is unscientific.

Reply
[-]johnlawrenceaspden00

I agree with both parts of this.

Reply
[-]johnlawrenceaspden00

Well, I don't think the big diagram is 'a good idea'. 'Making a big diagram' is a good idea. And all the techniques used to make it are good ideas. Similarly the map of England is not simple, but the idea of map-making and the techniques of surveying are.

As far as purpose goes, I think you're right.

We can ask, in evolutionary mode, "what if anything gave this a selective advantage?" or, relatedly, "why didn't this costly thing get selected out of existence?". And we can ask "what does this actually do?". What does talk of > purpose add beyond these?

If it were the case that talk of purpose couldn't be reduced to those things, and it were actually implying a fully intelligent mind, or some sort of intelligence to evolution beyond 'optimization process proceeding by local hill-climbing', then I'd get very suspicious indeed.

However, I just like teleological thinking. It's a bit like infinitesimals in analysis. They're not a bad way to get out quick and dirty results, but they can mislead. The careful man goes back and does the epsilon-delta thing as well. But I don't like to discard sources of intuition. We have so few!

And I think that a lot of the reason for mistrust of purpose is historical silliness like 'We die so that there will be room for the young'. That doesn't work at all for a sexual species. But that sort of thinking might be helpful for something undergoing group-selection. And pathogens, I think, often reproduce by cloning. One brave virus heading off to take out the bridge might well be genetically fulfilled by its sisters busy looting the cargo hold.

Reply
[-]gjm10

I don't think the big diagram is 'a good idea'. 'Making a big diagram' is a good idea.

That's a very reasonable distinction. But then -- bringing the analogy back to the motivating example of thyroid-y things -- why not say 'these symptoms are caused by deficiency of, or insensitivity to, thyroid hormones' and 'these symptoms are caused by [INSANELY COMPLEX EXPLANATION WE DON'T ACTUALLY HAVE GOES HERE]' are not 'good ideas' -- the good idea is something like 'figuring out what causes the symptoms'?

I mean, isn't that pretty much the level of abstraction that corresponds to "making a big diagram"?

I just like teleological thinking

Fair enough, I guess. It does seem that our brains are designed to ... ahem, I mean are pretty good at analysing things in terms of purposes, so thinking that way may be a useful hack. But you need to be awfully careful about it, as those historical sillinesses show.

(You can use infinitesimals just fine if you do it in terms of nonstandard analysis, which tells you exactly what you need to be careful about when doing so.)

Reply
[-]johnlawrenceaspden00

I want to back off from 'All the good ideas have turned out to be simple', I think that was a rhetorical flourish too far, and I think you were right to call bullshit.

I bet there are good ideas that aren't that simple, like e.g. the monster group or quantum chromodynamics. Neither of which I understand well enough to see whether they're actually simple, or inherently complex. But either way, it seems unlikely that the degree of complexity that fits happily in a human mind is also a limit on the complexity of ideas used in the construction of the world.

But I also want to defend: 'these symptoms are caused by deficiency of, or insensitivity to, thyroid hormones'

As a really simple idea that makes lots of predictions about real experiments that can be done.

Do you really think it isn't, or are we just arguing about 'All the good ideas have turned out to be simple'? If the latter, then you win. You were right and I was wrong. Thank you for the lesson!

Reply
[-]gjm10

As far as I'm concerned, we were just arguing about "All the good ideas have turned out to be simple".

(But if I was right and you were wrong, then you win because you've learned more than I have :-).)

Reply
[-]johnlawrenceaspden00

Indeed! I am properly grateful.

Reply
[-]gjm00

[EDITED to remove now-merely-confusing comments on something JLA has now removed from the parent comment.]

Reply
[-]ScottL70

I think that it would probably be a good idea to differentiate: ‘simple explanations’ and 'explanations that are based on simple rules'. See Fake Simplicity for a description of simple explanations. An example would be attributing all of the causality to some other entity, e.g. god. Explanations that are based on simple rules can sometimes also be easy to understand, but the way in which they are reached is rarely simple. They are grounded in extensive research and evidence.

Simple explanations can be dangerous because they are easy to believe. They are:

  • Easy to understand
  • Sometimes partly correct as they can be true some of the time even though they don’t describe the whole picture
  • Often overly broad so that they are hard to disprove
  • Etc.

I am just trying to saying that we should also be careful of simple explanations as well because they can be enticing. I would think that non-experts rarely have enough experience to reach explanations based on simple rules and will instead often just find simple explanations. This is because it is really hard or, perhaps, even impossible to find these simple rules without a lot of ground work. We often have to understand something intimately and deeply before we even begin to sense the undercurrent from the operation of these simple rules.

Here’s an extract from Feynman which is related:

The world is strange. The whole universe is very strange, but you see when you look at the details that the rules of the game are very simple – the mechanical rules by which you can figure out exactly what is going to happen when the situation is simple. It is like a chess game. If you are in a corner with only a few pieces involved, you can work out exactly what is going to happen, and you can always do that when there are only a few pieces. And yet in the real game there are so many pieces that you can't figure out what is going to happen – so there is a kind of hierarchy of different complexities. It is hard to believe. It is incredible! In fact, most people don't believe that the behavior of, say, me is the result of lots and lots of atoms all obeying very simple rules and evolving into such a creature that a billion years of life has produced. There is such a lot in the world. There is so much distance between the fundamental rules and the final phenomena that it is almost unbelievable that the final variety of phenomena can come from such a steady operation of such simple rules.

Reply
[-]johnlawrenceaspden00

For the Feynman quote, I don't think Feynman was a programmer! And Chaos was late in his life. Incomprehensibly complex behaviour from simple rules is not surprising to me.

Reply
[-]gjm50

Feynman certainly wasn't a professional programmer and I don't believe he spent much time writing software, but he worked for a while as a consultant to Thinking Machines Corporation and gave a (published, rather good) set of Lectures on Computation.

So if he doesn't sound like a programmer, it may be for reasons other than not being familiar with the insights that come from spending time making software.

(I don't think Feynman had any more difficulty than you in believing that simple rules produce very complex behaviour. I think he was trying to express how most people feel about it.)

Reply
[-]Gunnar_Zarncke30

And Feynman was very good at making people understand him. His lectures are well known for that.

Reply
[-]HungryHobo30

I thought I was reasonably comfortable with basic parallel programming... then I spent a few hours talking with a very smart woman who specialized in low level parallel programming(close to the silicon) and talking about the kind of things that can go wrong.

It's like clicking into an article expecting some light reading and finding yourself staring into a lovecraftian madness-educing abyss filled with 5-D monsters.

Reply
[-]Vaniver10

Something like dealing with synchronization and locking instead of just "and then we send this command to the mapper, and a lot of executors work in parallel"?

Reply
[-]HungryHobo50

No, I was comfortable with locking and the sort of stuff you'll routinely see in high level languages where they lock away the complexity behind abstraction.

I'm talking about the "fun" of physical cores which few sane programmers touch.

Though most programmers outside of chip companies can't touch those levels nowdays since even assembly is treated like a high-level language.

http://blog.erratasec.com/2015/03/x86-is-high-level-language.html#.VvQhiOaAnSg

Reply
[-]johnlawrenceaspden40

I had something of a similar experience writing software for a company that designed its own chips. I was writing in C, and occasionally, the programs had errors that I couldn't debug. At that point I'd dive into the Verilog for the chip and see how it was all wired up. And often that would be the problem.

It's a commonplace that the thing that determines the meaning of a computer program is itself a computer program. But it's still weird to get bitten by that fact when you're writing C.

The chip is a program, which determines the meaning of the machine code, which is a program produced by the C compiler, which is a program interpreting the program I'm writing. And whether you look at the chip as a compiled version of the verilog (compiled by another program, with silicon as the target) or as a logical program which you're proving things about in your head, or as a program being interpreted by a simulator, that's a lot of turtles.....

Reply
[-]ScottL10

Perhaps, 'laws' would have been a better word than 'rules'.

I was thinking of it more in terms of complexity. When things are looked at in isolation, it is much easier to see how the simple laws apply. But as things get more complex, we also need to figure out how the different systems interact and influence each other. This makes the simple laws harder to discern.

Simple systems have few components and their behavior is in all respects fully understandable and predictable. An example would be a solid ball falling under the action of gravity through air. This simple system consists of the ball, the air, and the gravitational force. Here we usually assume a single ball, constant acceleration of gravity, a viscous drag on the ball, and Newton’s laws. When making these assumptions, we arrive at very useful answers. We did, however, neglect many aspects. If, for example, we would ask how the behavior changes when we go from one ball, to two, to three, or even more balls that fall close to each other, our “Simple System” assumption fails. It is not sufficient to generalize from one ball’s behavior to many. Instead we need to consider the interaction of the balls through their self‐generated vortices.

Reply
[-][anonymous]00

I thought I was reasonably comfortable with basic parallel programming... then I spent a few hours talking with a woman who specialized in low level parallel programming(close to the silicon) and the kind of things that can go wrong.

It's like clicking into an article expecting some light reading and finding yourself staring into a lovecraftian madness-educing abyss filled with 5-D monsters.

[This comment is no longer endorsed by its author]Reply
[-]johnlawrenceaspden00

Yes, I think that's all really solid. I'm definitely going for a 'simple explanation' rather than 'an explanation based on simple rules'. It hasn't even occurred to me that I can describe what's going on mathematically.

And one should be very careful indeed of such ideas, especially if they have human consequences. Which is why I mentioned people who were probably trying to do their best to make the world a better place, who would probably have not been too pleased with how their ideas turned out.

But I keep thinking about atomism. Democritus worked it out thousands of years ago, from simple observations: 'By convention there is colour, by convention there is sweetness, in reality, only atoms and the void'.

The essence of the truth. What Feynman called the one fact that he would like to communicate to an ignorant civilization.

And yet it took two thousand years to prove. And the answer was found by looking carefully into a cup of tea (poetically speaking!).

I do wonder whether the Greeks would have worked it out, if they'd had a real go instead of just believing their a priori assumptions.

There are many such examples.

In my particular case (which I am not claiming is anything like as important, even if it is bang right!):

Easy to understand

I think so, but everyone seems to think it's absurd

Sometimes partly correct as they can be true some of the time even though they don’t describe the whole picture

That's what I'm claiming. We should take it seriously and have a look!

Often overly broad so that they are hard to disprove

A great danger. Luckily I've suggested a simple experiment that would refute the whole idea beyond doubt.

Reply
[-]The_Bird20

There's always the classic:

The lady down the street is a witch, she did it.

Reply
[-]johnlawrenceaspden20

It's a fair cop.

I would so like to leave it there, but in case anyone doesn't get it, it's a quote from 'Monty Python and the Holy Grail'.

And what I mean by it is: It's not a bad idea, if you believe that the world is ruled by good and evil powers, that some people might serve evil.

The problem is that it doesn't help you find the evil-servers.

You need a witch-test.

And you need to check very carefully that it works! Or you will do immense harm. (I am looking at you, TSH)

We didn't stop torturing and burning witches because we suddenly became kind. If Satan, and witches, existed, then they would deserve every bit of it. (Obviously I speak of the mediaeval conception of witches, not the modern pagans, who are awfully nice people who care about trees and knitting)

We stopped torturing people when we realised that you can get anyone to confess to anything by torture. (I believe that experiments were conducted.)

And now, we don't believe in the Evil One, so we don't even think witches exist. (Leaving the Wicca people to play their beautiful harmless games which everyone approves of)

Reply
[-]Lumifer10

You need a witch-test.

Which Sir Bedevere provides :-) And after some critical inquiry, no less!

Reply
[-]Jiro10

People seem to hate it because it is so simple, and makes so many predictions, most of which are terrifying.

And it is obviously false! Of course medicine has tried using thyroid supplementation to fix 'tired all the time'. It doesn't work!

That's not a coincidence.

The fact that an idea is simple and makes predictions with big implications (but supposedly has been ignored) is, all by itself, evidence that the idea is false.

Reply
[-]johnlawrenceaspden00

Oh, I absolutely agree! What's irritating me is that that now seems to be the only reason to believe that my thyroid craziness is false.

(medical science even remotely competent) => (idea is false)

As they say, one man's modus ponens is another man's modus tollens.

It turns out I was a bit too fast with:

And it is obviously false! Of course medicine has tried using thyroid supplementation to fix 'tired all the time'. It doesn't work!

This paper:

Clinical Response to Thyroxine Sodium in Clinically Hypothyroid but Biochemically Euthyroid Patients G. R. B. SKINNER MD DSc FRCPath FRCOG, D. HOLMES, A. AHMAD PhD, J. A. DAVIES BSc and J. BENITEZ MSc Vaccine Research Trust, 22 Alcester Road, Moseley, Birmingham B13 8BE, UK

Is the report of a man who tried the blindingly obvious thing (just treating CFS patients with thyroxine, and raising the dose until they got better), which I've been saying all along can't possibly work because it is literally unbelievable that medical science can have missed this.

Apparently it worked a treat. This is from 2000. If it's been followed up (apart from the misinterpreted Scottish trial that I referenced in the first post in support of the idea), I can't find it.

I mean, look at all those letters: G. R. B. SKINNER MD DSc FRCPath FRCOG

This is not a homeopath. I am at an absolute loss to explain why this has been ignored.

Of course, like everyone else involved, Gordon Skinner is dead. What kills these people? Thyroxine overdose? What silences them? Alien lizard conspiracy pyramids? How would that even work?

Reply
[-]Jiro10

This paper:

Beware the man of one study. One study, or even a couple of studies, can prove pretty much anything simply because there are so many studies that there will be lots of statistical anomalies.

Reply
[-]johnlawrenceaspden00

But I am the man of two studies! (The scottish trial proves that thyroxine is less bad for their patient group than for healthy controls). And as far as I can tell, there are no other studies.

And an absolute fuckton of anecdotal evidence from the web and from alternative medicine, and from my own personal experience.

Now, I do not actually believe this works without a solid PCRT with a p-value much lower than 0.05, but after seeing all this, I certainly predict that a properly designed PCRT will come up with the goods.

And it affects the health of millions! These people are asleep on the job.

Reply
[-]Jiro-10

But I am the man of two studies!

Two counts as "a couple of studies".

And an absolute fuckton of anecdotal evidence from the web and from alternative medicine, and from my own personal experience.

That's equivalent to "no good evidence". You can't make up for quality with quantity here.

Reply
[-]johnlawrenceaspden00

Now we are in actual disagreement. Two results to p<0.05 out of a total of two relevant experiments is pretty good.

If there were a further four with 'no effect' I might be less impressed. But there aren't, unless they're in a file drawer somewhere.

You can't make up for quality with quantity here.

Where is it written that no amount of bad evidence can add up to good evidence? Evidence is evidence, Bayes-wise.

I say that the sun rises every morning. There has been no PCRT. Am I likely mistaken?

Reply
[-]Lumifer10

Two results to p<0.05 out of a total of two relevant experiments is pretty good.

You are misreading the situation. You have two p<0.05 results out of two published studies and out of a total of no one knows how many relevant experiments.

Where is it written that no amount of bad evidence can add up to good evidence?

Thus spake the Book of Selection Bias. And the Book of Making Shit Up sagely nodded and said "Verily this is so".

Reply
[-]johnlawrenceaspden00

Well, fine, but in order to get two p<0.05 results by chance if there's no effect, there'd have had to be roughly forty failed experiments done and never mentioned. Is that likely?

But the question's easy to settle if that's what we really think might have happened. Just pre-register and replicate one.

I really think that if there were enough interest in the question that it had prompted forty failures, someone would have had the wit to do that. Don't you?

I mean, we've got an a priori plausible hypothesis, lots of evidence for, some of it solid, most of it weak, no evidence against. There would have to be one hell of a filter somewhere to justify ignoring that. Wouldn't there?

Reply
[-]Lumifer10

Well, fine, but in order to get two p<0.05 results by chance if there's no effect, there'd have had to be roughly forty failed experiments done and never mentioned. Is that likely?

First, that forty number comes from the spherical-cow land where everything is independent, normally distributed, gardens of forking paths do not exist, etc. etc. Look at the replication crisis in psychology which has been getting a lot of press recently. They have dozens of papers showing highly significant results for some effect which, as it turns out now, does not exist.

Second, consider the incentives. Say, you ran a small trial, got zero results, what are you going to do with it? No journal will be very excited about the "we tried a weird thing and it didn't work" paper. Or, say, you got negative results, your patients started dying. Would you be terribly interested in writing up "we tried a weird thing and ended up killing some people" results?

Wouldn't there?

You need to convince not me, but people who are good at writing grant proposals and passing ethics boards :-/

Reply
[-]johnlawrenceaspden00

You need to convince not me, but people who are good at writing grant proposals and passing ethics boards :-/

Sure, but I have now managed to convince myself, so I am practising. Please be the most evil opponent you can be!

Second, consider the incentives.

Incentives-wise, doctors are regularly getting struck off for believing and practising this, all over the world. And the NICE guidelines specifically say that thyroxine is not to be used for the treatment of CFS. You'd think they'd be overjoyed to have a reference to quote when striking people off/writing guidelines.

Would you be terribly interested in writing up "we tried a weird thing and ended up killing some people" results?

Well, I think that's exactly what the Scottish GPs thought they were writing up! They found 'no difference from placebo' in their patient group, and 'harmful' in their control group.

Where we differ is that I think that means that they must actually have done a fair bit of good in their patient group (specifically, in that portion of their patient group who actually had type 2 hypothyroidism, and for whom 100mg thyroxine/day was roughly the right amount. According to Skinner, that would have been too much for many of them, and too little for many others).

Reply
[-]PipFoweraker10

Following simple ideas or explanations that are mostly right will still give me good outcomes in a plurality of iterations.

I don't have infinite time to carefully consider those ideas. Being a standardly incompetent human at many things, my ability to discern truthiness by looking is subject to error.

What options do I have for differentiating between simple explanations that are correct and simple explanations that are only mostly correct, and then figuring out whether the latter are worth investigating for corner-casery / quackery / etc?

Reply
[-]johnlawrenceaspden20

Well, pretty much by definition you'll do well following the mostly correct ideas. And when things go wrong, you'll have produced exactly the sort of surprise that (gets physicists/should get anyone who calls themselves a scientist) to say 'Oooh, that's interesting'.

So if you're lazy, go with the obvious (the world is flat), and only trouble yourself to think if you're (a) curious or (b) confounded by your bad anticipations.

Generally speaking 'What the wise ones say is (likely) true' is not a bad heuristic if you don't have the time or reason to be interested.

Reply
[-][anonymous]10

Simple Ideas That Explain Everything But Are Wrong?

'incentives'

Reply
[-]johnlawrenceaspden50

Clarity, you are unclear.

Reply
[-]Elo10

I think clarity is saying: a simple idea that explains all human motivation for ever is "incentives". Although I personally am not sure that it is wrong.

Reply
[-]johnlawrenceaspden00

Thank you. I worry that I am becoming some sort of scientistic mystic. Am I as unclear as Clarity?

Reply
[-]Elo10

If you are unclear; at least you do it with more words.

Simple Ideas That Explain Everything But Are Wrong? 'incentives'

is a very short post; makes it hard for people who don't get it to catch up. (no offence: I believe he meant it to be short to follow the rule of "simple ideas")

Reply
[-]gjm00

No.

Reply
[-]johnlawrenceaspden10

OK, I get it, thanks Elo.

If you believe that people don't respond to incentives, then we don't live in the same world.

If you believe that people only respond to monetary incentives, then we don't live in the same world.

The job is to work out which world we live in. Incentives is probably a big part of that!

Reply
[-][anonymous]10

Incentives in and of themselves explain a relationship between the incentive and a referent thing incentivised by it. It's like saying 'force' in physics, which is incompatible within a framework of mutually exclusive but comprehensively exhaustive concepts which make up a interdisciplinary modelling kit for the world, but people were cool with it (till Einstein changed that) because of their narrow focus.

Reply
[-]johnlawrenceaspden10

I'm too stupid to understand this. Could you dumb it down for me? Are you saying that the idea of force in physics was not simple, or good, or useful, or that it didn't explain a lot of things? Or have I got the wrong end of your stick?

Reply
[-][anonymous]00

Force in physics was simple, good and useful, but less simply, good and useful that relativity, which doesn't require 'force' as an explanatory mechanism because it doesn't explain a lot of things.

Reply
Moderation Log
More from johnlawrenceaspden
View more
Curated and popular this week
53Comments