Sure, this experiment is evidence against 'all fat, tired people with dry hair get better with thryoxine'. No problem there.

Okay, but you said it was evidence in favor of your own hypothesis. That’s what my question was about.

Yes, it is kind of odd isn't it? One of the pills apparently made them a bit unwell, and yet they couldn't tell which one. I notice that I am confused.

Suppose they’re measuring on a 10-point scale, and we get ordered pairs of scores for time A and time B. One person might have 7 and 6, another has (4,3), another has (5,6), then (9,7), (7,7), (4,5), (3,2)...Even if they’re aware of their measurements (which they might not be), all sorts of things affect their scores and it’s unlikely that any one person would be able to make a conclusion. You’re basically asking an untrained patient to draw a conclusion from an n of 1.

But that's awful! Once, there was a diagnostic method, and a treatment that worked fine, that everyone thought was brilliant. Then they invented a test, which is very clever, and a good test for what it tests, and the result of that is that lots of people are ill and don't get the treatment any more and have to suffer horribly and die early.

There are several assumptions here that I think are probably incorrect, the biggest being the causal link between introducing the test and people suffering. But what I described before is just the application of reductionism to better distinguish between disease states based on their causal mechanism.

If that's normal then there's something badly wrong with normal. A new way of measuring things should help!

Sometimes, but replacing an objective measurement with a subjective one isn’t usually a step forward.

Seriously, if 'start off with low doses and keep raising the dose until you get a response' is inaccessible to testing, then something is broken.

Problems with this include: you can’t justify the parameters of the dose increase, you still have to agree on how to measure the response, and you also have a multiple testing issue. It isn’t inaccessible, but it’s a complication (potentially a major one), and that’s just in the abstract. Practically, in any one situation there might be another half dozen issues that wouldn’t be apparent to anyone who isn’t an expert.

But in fact, just 'low basal metabolic rate in CFS' would be good evidence in favour, I think. We can work out optimal treatments later.

Not knowing anything about the subject, I would expect to observe a low basal metabolic rate in CFS regardless of its ultimate cause or causes.

At that point, we're all post-modernists aren't we? The truth is socially determined.

No, it just means we put very little weight on individual studies. We don’t pay much attention to results that haven’t been replicated a few times, and rely heavily on summaries like meta-analyses.

Science is not unreliable...

You’re talking about the overall process and how science moves in the direction of truth, which I agree with. I’m talking on the level of individual papers and how our current best knowledge may still be overturned in the future. But you can leave out “just like..wisdom” from the paragraph without losing the main points.

There's at least a possibility here that medical science is getting beaten hollow by chiropractors and quack doctors and internet loonies, none of whom have any resources or funding at all.

The alt med people have a lot of funding. It’s a multi-billion-dollar industry.

Even the possibility is enough to make me think that there's something appallingly badly wrong with the methods and structure of medical science.

A few things, not just one, but it’s the best we have at the moment.