private_messaging comments on Too good to be true - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (119)
I realize that, and I've already pointed out why the difference in rates is not going to be that large & that your cite does not explain the excess significance in their sample.
Doesn't matter that much. Power, usually quite low, sets the upper limit to how many of the results should have been positive even if we assume every single one was testing a known-efficacious drug (which hypothesis raises its own problems: how is that consistent with your claims about the language bias towards publishing cool new results?)
So? I don't care why the Russian literature is biased, just that it is.
Yes, but toxicology studies being done by industry is not aimed at academic publication, and the ones aimed at academic publication have the usual incentives to find something and so are part of the overall problem.
Huh? The paper finds that acupuncture study rates vary by region. USA/Sweden/Germany 53/59%/63%, China/Japan/Taiwan 100% etc
How much have you looked? There's plenty of acupuncture centres in the USA despite a relatively low acupuncture success rate.
Does a fish notice water? But fine, maybe you don't, feel free to supply your own example of Russian pseudoscience and traditional medicine. I doubt Russian science is a shining jewel of perfection with no faults given its 91% acupuncture success rate (admittedly on a small base).
Not sure that's a good example, as Wikipedia seems to disagree about homebrew phage therapy not being applied: https://en.wikipedia.org/wiki/Phage_therapy#History
Anyway,
How do you see the unseen? Unless someone has done a large definitive RCT, how does one ever prove that a result was bogus? Nobody is ever going to take the time and resources to refute those shitty animal experiments with a much better experiment. Most scientific findings never gets that sort of black-and-white refutation, it just gets quietly forgotten and buried, and even the specialists don't know about it. Most bad science doesn't look like Lysenko. Or look at evidence-based medicine in the West: rubbish medicine doesn't look like a crazy doc slicing open patients with a scalpel, it just looks like regular old medicine which 'somehow' turns up no benefit when rigorously tested and is quietly dropped from the medical textbooks.
To diagnose bad science, you need to look at overall metrics and indirect measures - like excess significance. Like 91% of acupuncture studies working.
If you want to persist in your mythical ideas regarding western civilization by postulating what ever you need and making shit up, there's nothing I or anyone else can do about it.
Your study is making a more specific claim than mere bias in research, it's claiming bias in one particular direction.
The point is that the SU was, mostly, using antibiotics (once production was set up, i.e. from some time after ww2).
Well, and there wasn't a plenty in the soviet union despite supposedly higher success rate.
If you don't know correct rate you can't tell which specific rate is erroneous. It's not realistically possible to construct a blind study of acupuncture, so, unlike, say, homoeopathy, it is a very shitty measure of research errors.
I really doubt that 91% of Russian language acupuncture studies published in Soviet Union found a positive effect (I dunno about 1991-1998 Russia, it was fucked up beyond belief at that time), and I don't know how many studies should have found a positive effect (followed by a note that more adequate blinding must be invented to study it properly).
And we know that what ever was the case there was no Soviet abandonment of normal medicine in favour of acupuncture - the system somehow worked out ok in the end.
That's not a reply to what I wrote.
Yes, that's what a bias is. A systematic tendency in one direction. As opposed to random error.
And before that, they were using phages despite apparently pretty shaky evidence it was anything but a placebo. That said, pointing out the systematic bias of Russian science (among many other countries, and I'm fascinated, incidentally, how the only country you're defending like this is... your own. No love for Korea?) does not commit me to the premise that phages do or not work - you're the one who brought them up as an example of how excellent Russian science is, not me.
How many are there now? Shouldn't you have looked that up?
Difference in rates is prima facie evidence of bias, due to the disagreement. If someone says A and someone else says not-A, you don't need to know what A actually is to observe the contradiction and know at least one party is wrong.
Yes it is.
And naturally, you have not looked for anything on the topic, you just doubt it.
Strawman. No country engages in 'abandonment of normal medicine' - if you go to China, do you only find acupuncturists? Of course not. The problem is that you find acupuncturists sucking up resources in dispensing expensive placbeos and you find that the scientific community is not strong enough to resist the cultural & institutional pressures and find that acupuncture doesn't work, resulting in real working medicine being intermeshed with pseudomedicine.
Fortunately, normal medicine (after tremendous investments in R&D and evidence-based medicine) currently works fairly well and I think it would take a long time for it to decay into something as overall bad as pre-modern Western medicine was; I also think some core concepts like germ theory are sufficiently simple & powerful that they can't be lost, but that would be cold comfort in the hypothetical cargo cult scenario ('good news: doctors still know what infections are and how to fight epidemics; bad news: everything else they do is so much witch-doctor mumbojumbo based on unproven new therapies, misinterpretations of old therapies which used to work, and traditional treatments like acupuncture').
I was referring to your other comment.
That's the one I know most about, obviously. I have no clue about what's going on in China, Korea, or Japan.
Look, it doesn't matter if phages work or don't work! The treatment, in favour of which there would be strong bias, got replaced with another treatment, which would have been biased against. Something that wouldn't have happened if science systematically failed to work in such an extreme and ridiculous manner. I keep forgetting that I really really need to spell out any conclusions when arguing with you. It's like you're arguing that a car is missing the wheels but I just drove here on it.
Besides, the 90%+ proportion of positive results is also the case in the west
(also, in the past we had stuff like lobotomy in the west)
So why do you think your defense would not apply equally well (or poorly) to them? What's the phage of China?
Oh wow. What a convincing argument. 'Look, some Russians once did this! Now they do that! No, it doesn't matter if they were right or wrong before or after!' Cool. So does that mean I get to point to every single change of medical treatment in the USA as evidence it's just peachy there? 'Look, some Americans once did lobotomy! Now they don't! It doesn't matter if lobotomies work or don't work!'
You didn't drive shit anywhere.
That's on a different dataset, covering more recent time periods, which, as the abstract says, still shows serious problems in East Asia (compromised by relatively small sample: trying to show trends in 'AS' using 204 studies over 17 years isn't terribly precise compared to the 2627 they have for the USA) with the latest data being 85% vs 100%. And 100% significance is a ceiling, so who knows how bad the East Asian research has actually gotten during the same time period Western numbers continue to deteriorate...
Unless A contains indexicals that point to different things in the two cases.
(Maybe Asian acupunturists are better than European ones, or maybe East Asians respond better to acupuncture than Caucasians for some reason, or...)
( I'm not saying that this is likely, just that it's possible.)
You are trying to pull a "everyone and me is against you" stunt against Gwern? Do you have any idea how dumbfoundingly absurd this would sound to most of those of the class "anyone else" who happens to see this exchange?