Sure, this experiment is evidence against 'all fat, tired people with dry hair get better with thryoxine'. No problem there.

Okay, but you said it was evidence in favor of your own hypothesis. That’s what my question was about.

Yes, it is kind of odd isn't it? One of the pills apparently made them a bit unwell, and yet they couldn't tell which one. I notice that I am confused.

Suppose they’re measuring on a 10-point scale, and we get ordered pairs of scores for time A and time B. One person might have 7 and 6, another has (4,3), another has (5,6), then (9,7), (7,7), (4,5), (3,2)...Even if they’re aware of their measurements (which they might not be), all sorts of things affect their scores and it’s unlikely that any one person would be able to make a conclusion. You’re basically asking an untrained patient to draw a conclusion from an n of 1.

But that's awful! Once, there was a diagnostic method, and a treatment that worked fine, that everyone thought was brilliant. Then they invented a test, which is very clever, and a good test for what it tests, and the result of that is that lots of people are ill and don't get the treatment any more and have to suffer horribly and die early.

There are several assumptions here that I think are probably incorrect, the biggest being the causal link between introducing the test and people suffering. But what I described before is just the application of reductionism to better distinguish between disease states based on their causal mechanism.

If that's normal then there's something badly wrong with normal. A new way of measuring things should help!

Sometimes, but replacing an objective measurement with a subjective one isn’t usually a step forward.

Seriously, if 'start off with low doses and keep raising the dose until you get a response' is inaccessible to testing, then something is broken.

Problems with this include: you can’t justify the parameters of the dose increase, you still have to agree on how to measure the response, and you also have a multiple testing issue. It isn’t inaccessible, but it’s a complication (potentially a major one), and that’s just in the abstract. Practically, in any one situation there might be another half dozen issues that wouldn’t be apparent to anyone who isn’t an expert.

But in fact, just 'low basal metabolic rate in CFS' would be good evidence in favour, I think. We can work out optimal treatments later.

Not knowing anything about the subject, I would expect to observe a low basal metabolic rate in CFS regardless of its ultimate cause or causes.

At that point, we're all post-modernists aren't we? The truth is socially determined.

No, it just means we put very little weight on individual studies. We don’t pay much attention to results that haven’t been replicated a few times, and rely heavily on summaries like meta-analyses.

Science is not unreliable...

You’re talking about the overall process and how science moves in the direction of truth, which I agree with. I’m talking on the level of individual papers and how our current best knowledge may still be overturned in the future. But you can leave out “just like..wisdom” from the paragraph without losing the main points.

There's at least a possibility here that medical science is getting beaten hollow by chiropractors and quack doctors and internet loonies, none of whom have any resources or funding at all.

The alt med people have a lot of funding. It’s a multi-billion-dollar industry.

Even the possibility is enough to make me think that there's something appallingly badly wrong with the methods and structure of medical science.

A few things, not just one, but it’s the best we have at the moment.

Sure, this experiment is evidence against 'all fat, tired people with dry hair get better with thryoxine'. No problem there.

Okay, but you said it was evidence in favor of your own hypothesis. That’s what my question was about.

Yes, it is kind of odd isn't it? One of the pills apparently made them a bit unwell, and yet they couldn't tell which one. I notice that I am confused.

Suppose they’re measuring on a 10-point scale, and we get ordered pairs of scores for time A and time B. One person might have 7 and 6, another has (4,3), another has (5,6), then (9,7), (7,7), (4,5), (3,2)...Even if they’re aware of their measurements (which they might not be), all sorts of things affect their scores and it’s unlikely that any one person would be able to make a conclusion. You’re basically asking an untrained patient to draw a conclusion from an n of 1.

But that's awful! Once, there was a diagnostic method, and a treatment that worked fine, that everyone thought was brilliant. Then they invented a test, which is very clever, and a good test for what it tests, and the result of that is that lots of people are ill and don't get the treatment any more and have to suffer horribly and die early.

There are several assumptions here that I think are probably incorrect, the biggest being the causal link between introducing the test and people suffering. But what I described before is just the application of reductionism to better distinguish between disease states based on their causal mechanism.

If that's normal then there's something badly wrong with normal. A new way of measuring things should help!

Sometimes, but replacing an objective measurement with a subjective one isn’t usually a step forward.

Seriously, if 'start off with low doses and keep raising the dose until you get a response' is inaccessible to testing, then something is broken.

Problems with this include: you can’t justify the parameters of the dose increase, you still have to agree on how to measure the response, and you also have a multiple testing issue. It isn’t inaccessible, but it’s a complication (potentially a major one), and that’s just in the abstract. Practically, in any one situation there might be another half dozen issues that wouldn’t be apparent to anyone who isn’t an expert.

But in fact, just 'low basal metabolic rate in CFS' would be good evidence in favour, I think. We can work out optimal treatments later.

Not knowing anything about the subject, I would expect to observe a low basal metabolic rate in CFS regardless of its ultimate cause or causes.

At that point, we're all post-modernists aren't we? The truth is socially determined.

No, it just means we put very little weight on individual studies. We don’t pay much attention to results that haven’t been replicated a few times, and rely heavily on summaries like meta-analyses.

Science is not unreliable...

You’re talking about the overall process and how science moves in the direction of truth, which I agree with. I’m talking on the level of individual papers and how our current best knowledge may still be overturned in the future. But you can leave out “just like..wisdom” from the paragraph without losing the main points.

There's at least a possibility here that medical science is getting beaten hollow by chiropractors and quack doctors and internet loonies, none of whom have any resources or funding at all.

The alt med people have a lot of funding. It’s a multi-billion-dollar industry.

Even the possibility is enough to make me think that there's something appallingly badly wrong with the methods and structure of medical science.

A few things, not just one, but it’s the best we have at the moment.

Hmmm

This piece claims that

...we face a replication crisis in the field of biomedicine, not unlike the one we’ve seen in psychology but with far more dire implications. Sloppy data analysis, contaminated lab materials, and poor experimental design all contribute to the problem.

...Freedman and his co-authors guessed that fully half of all results rest on shaky ground, and might not be replicable in other labs. These cancer studies don’t merely fail to find a cure; they might not offer any useful data whatsoever.

What, you doubt Brussels which already saved the Europeans from the horrors of mis-curved bananas? X-0

Just buy a whole pig and eat it snout to tail :-)

I think it's important to have a more global vision of the problem. Knowing what the situation is in different countries could be a start. Post it as replies to this comment, with the country you live in, and the situation there (the best would be to ask one or more doctors about it, in order to be sure we can trust it). Personally, I live in France. Here, desiccated thyroid isn't sold anymore to everybody. According to two doctors I know, it's because bad things used to happen back in the days where it was completely accessible (in the 60' I believe). I am still researching informations about the hypothesis itself in my country, I'll post it if I learn anything. Thank you for your replies by advance!

Well, 'hypothyroidism' was a very difficult and polymorphic badger in its day.

Isn't it still "its day"?

Think of it this way. There is a set of people with some clinical symptoms which look maybe-possibly like hypothyroidism. There is a another set of people with abnormal TSH. These sets partially intersect and form three subsets. Subset one is the intersection: people with both clinical symptoms and abnormal TSH. They are a clear case and there are no problems here. Subset two is abnormal TSH and absence of clinical symptoms. We interpret that as thyroid gland falling apart and expect clinical symptoms to appear in the near future. We are not concerned with people either.

Subset three is the one you are interested in: people with normal TSH and clinical symptoms. What about them? Well, as you mention diagnosing hypothyroidism solely on the basis of clinical symptoms is difficult. So in this subset some but not all people will have a thyroid malfunction, and some will have other problems, maybe instead or maybe in addition to thyroid issues.

By the way, the people who you insist on calling "fat, tired, and with dry skin" are in subset three. They exhibit clinical symptoms of hypothyroidism.

Your suggestion is that we give some dessicated thyroid to subset three and see if it helps. Well, it's pretty clear that it will help some people and will not help other people (for example, those fat and tired ones). However that is true of many medical interventions.

For example, there are probably males in subset three with low testosterone. So giving testosterone to subset three males will also help some people and not help others. There also probably people with low-grade systemic infections in there. Giving broad-spectrum antibiotics to subset three might well help some people and not help others. There are likely people with autoimmune disorders there...

Basically, if you have little idea about what's wrong, trying a variety of drugs hoping for a lucky hit is not necessarily a horrible strategy (depends on the side-effects of the drugs and the consequences of doing nothing), but it's not much advancement from the good old times.

Let me quote from West Hunter:

Back in the good old days, Charles II, age 53, had a fit one Sunday evening, while fondling two of his mistresses.

Monday they bled him (cupping and scarifying) of eight ounces of blood. Followed by an antimony emetic, vitriol in peony water, purgative pills, and a clyster. Followed by another clyster after two hours. Then syrup of blackthorn, more antimony, and rock salt. Next, more laxatives, white hellebore root up the nostrils. Powdered cowslip flowers. More purgatives. Then Spanish Fly. They shaved his head and stuck blistering plasters all over it, plastered the soles of his feet with tar and pigeon-dung, then said good-night.

Tuesday. ten more ounces of blood, a gargle of elm in syrup of mallow, and a julep of black cherry, peony, crushed pearls, and white sugar candy.

Wednesday. Things looked good:: only senna pods infused in spring water, along with white wine and nutmeg.

Thursday. More fits. They gave him a spirituous draft made from the skull of a man who had died a violent death. Peruvian bark, repeatedly, interspersed with more human skull. Didn’t work.

Friday. The king was worse. He tells them not to let poor Nelly starve. They try the Oriental Bezoar Stone, and more bleeding. Dies at noon.

Well, 'hypothyroidism' was a very difficult and polymorphic badger in its day.

Isn't it still "its day"?

Think of it this way. There is a set of people with some clinical symptoms which look maybe-possibly like hypothyroidism. There is a another set of people with abnormal TSH. These sets partially intersect and form three subsets. Subset one is the intersection: people with both clinical symptoms and abnormal TSH. They are a clear case and there are no problems here. Subset two is abnormal TSH and absence of clinical symptoms. We interpret that as thyroid gland falling apart and expect clinical symptoms to appear in the near future. We are not concerned with people either.

Subset three is the one you are interested in: people with normal TSH and clinical symptoms. What about them? Well, as you mention diagnosing hypothyroidism solely on the basis of clinical symptoms is difficult. So in this subset some but not all people will have a thyroid malfunction, and some will have other problems, maybe instead or maybe in addition to thyroid issues.

By the way, the people who you insist on calling "fat, tired, and with dry skin" are in subset three. They exhibit clinical symptoms of hypothyroidism.

Your suggestion is that we give some dessicated thyroid to subset three and see if it helps. Well, it's pretty clear that it will help some people and will not help other people (for example, those fat and tired ones). However that is true of many medical interventions.

For example, there are probably males in subset three with low testosterone. So giving testosterone to subset three males will also help some people and not help others. There also probably people with low-grade systemic infections in there. Giving broad-spectrum antibiotics to subset three might well help some people and not help others. There are likely people with autoimmune disorders there...

Basically, if you have little idea about what's wrong, trying a variety of drugs hoping for a lucky hit is not necessarily a horrible strategy (depends on the side-effects of the drugs and the consequences of doing nothing), but it's not much advancement from the good old times.

Let me quote from West Hunter:

Back in the good old days, Charles II, age 53, had a fit one Sunday evening, while fondling two of his mistresses.

Monday they bled him (cupping and scarifying) of eight ounces of blood. Followed by an antimony emetic, vitriol in peony water, purgative pills, and a clyster. Followed by another clyster after two hours. Then syrup of blackthorn, more antimony, and rock salt. Next, more laxatives, white hellebore root up the nostrils. Powdered cowslip flowers. More purgatives. Then Spanish Fly. They shaved his head and stuck blistering plasters all over it, plastered the soles of his feet with tar and pigeon-dung, then said good-night.

Tuesday. ten more ounces of blood, a gargle of elm in syrup of mallow, and a julep of black cherry, peony, crushed pearls, and white sugar candy.

Wednesday. Things looked good:: only senna pods infused in spring water, along with white wine and nutmeg.

Thursday. More fits. They gave him a spirituous draft made from the skull of a man who had died a violent death. Peruvian bark, repeatedly, interspersed with more human skull. Didn’t work.

Friday. The king was worse. He tells them not to let poor Nelly starve. They try the Oriental Bezoar Stone, and more bleeding. Dies at noon.

If none of the patients had had any sort of thyroid problem, I'd have expected it to be equally bad for everyone.

I’m talking about conservation of expected evidence. If X is positive evidence, then ~X is negative evidence. An experiment only supports a hypothesis if it was possible for it to come out another way that refutes it. And if an experiment that could have supported the hypothesis actually didn’t, then it’s evidence against.

What makes me think that they felt bad on thyroxine is table 2, where all the 'self-reported' psychological scores have got worse from thyroxine. In particular p=0.007 for the decline in Vitality. Since, as you point out, they really didn't know which was which, it's hard to see how they could have faked that.

Terminology then. When you said “Thyroxine is very strongly disliked by the healthy controls (they could tell it from placebo and hated it),” it suggests they could identify the active treatment.

Absolutely this treatment is harmful to healthy people.

The people in the study had symptoms. Even if you think their symptoms were mild or unrepresentative, you shouldn’t call them healthy. It’s fair to extend the conclusion to cover people without those symptoms, but I think that’s an important difference.

Yes, but that does mean that anything that needs careful dose control will get rejected.

It’s more that you need an easily followed protocol. Anything else, especially anything subjective, is unlikely to be practically feasible, and will probably not be reproducible.

The TSH test replaced that around 1970. But they never seem to have checked that clinical and biochemical diagnoses detected the same things, and after that there was the slow emergence of all sorts of nasty diseases that look very like hypothyroidism in the clinical sense but have normal TSH.

This is normal. Clinical presentations often have many causes, which makes it almost impossible to progress. Eventually we break them down based on their causal mechanisms so we can treat them individually. Each time we find a new cause, some of the cases will be left unexplained.

These are the only ones I can find through google scholar / pubmed. That in itself is really surprising and one of the things I can't explain! Why has such an obvious thing not been ruled out?

There are a lot of interesting hypotheses competing for resources, and we have to decide which ones are worth considering. I can’t say what the reason might be here, but there are a lot of possibilities. For example, it might not be possible to design a study like the one you want that could effectively answer the question.

Really? Forty years of experience in treating patients is less valuable than a single anecdote published in a journal? Really?

Yes. Expert opinion (i.e., the opinion of individual experts, not expert consensus) is the lowest level because you can find an expert to support pretty much any proposition that isn’t obviously ridiculous, and sometimes even if it is. In fact, this is true higher in the hierarchy as well, which is why we use syntheses of evidence so much. I can’t stress this enough: in biology, you can use peer-reviewed evidence to make plausible arguments for arbitrary hypotheses.

All the rest of it is anecdotal, from alternative sources, but there's a mountain of it.

The point of evidence-based medicine is that perceptions are unreliable. That includes the perceptions we call clinical experience (which once said that bloodletting was an important medical treatment). Keep in mind that doctors aren’t scientists and usually don’t even qualify as experts. EBM is unreliable too, but less so, just like science is unreliable but is still better than ancestral wisdom.

The TSH test ruling out hypothyroidism is expert opinion. Its reliability is unfounded dogma.

This sounds like you’re saying the TSH test doesn’t actually measure TSH, but I think you mean to say you disagree with the conclusions that it’s used for. But since hypothyroidism is defined as low thyroid hormone levels, some of this will be a dispute over definitions.

I can't find any evidence for it as the sole measure of thyroid system function at all.

I don’t think anyone who understands it would say it is. It measures TSH levels, and the question is what we do with that measurement. But we’re often limited by what we’re able to (easily) measure, and it might even be the only objective measurement we have.

Something like 'inadequate thyroid-hormone-mediated regulation of metabolism'.

That's wonderfully vague. I bet I can diagnose half the population with having "inadequate" regulation.

A definition should allow easy classification of observed phenomena into two classes: "fits the definition" and "doesn't fit the definition". This one... struggles.

we might actually need to understand how it works to treat

Yes, I have such a suspicion, too.

'acquired generalised hormone resistance disorders'

And this I can probably diagnose 90% of the population with? See above.

The meta issue is whether you want to medicalise deviations from the theoretical optimum. On the one hand, sure, it's nice to move closer to the optimum, on the other hand this means that no one is "healthy", everyone is "sick" and under care of doctors.