It is fairly terrifying that the term "evidence-based medicine" exists because that implies that there are other kinds.
LessWrong is a non-evidence-based method of teaching rationality. We don't have good evidence that someone will get more rational after reading the sequences.
You can make a reasonable theoretic argument that people will get more rational. You don't have the kind of evidence that you need for a EBM-treatment. In most domains where we make choices in our lives you don't follow pratices that are supported by evidence from peer-reviewed trials.
You don't get a haircut from a barber who practices evidence-based barbering. Even the people who pay a lot of money for their haircuts don't. Reading scientific papers just isn't the only way to gather useful knowledge.
The term evidence-based medicine comes from a http://jama.jamanetwork.com/article.aspx?articleid=400956 published in 1992.
It says:
Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research.
I wouldn't want someone to practice open-heart surgery on me based on his intuition but I don't see a problem with taking a massage from someone who read no scientific papers but who let's themselves be guided by his intuition and who has a positive track record with other patients.
Doctors have had a much longer time than LessWrong to get their act together.
The idea of teaching people to think better isn't new. Aristoteles also tried to teach a form of rationality. But even if the idea would be radically new, why would that matter?
Why should newer ideas be subject to a lower standard of evidence? Fairness? If you want to know the truth fairness has no place.
Let's look at another example: Romantic courtship. Do you practice evidence-based courtship, when you seek a fulfilling relationship with a woman? Would you say there no other form of courtship besides evidence-based courtship?
Most humans don't practice evidence-based courtship. Sometimes courtship doesn't work out. You can blame it on couple not being familiar with the scientific papers that are published on the subject of human courtship.
Nobody has shown that given the couple those scientific papers improves their relationship changes. Nobody has shown with EBM like evidence standards that doctors who are in touch with the scientific literature archieve better health outcomes for their patients.
That doesn't mean to me that EBM has no place, but I don't see a reason to reject any approach to increase...
Once MetaMed has been paid for and done a literature search on a given item, will that information only be communicated to the individual who hired them, or will it be made more widely available?
Once MetaMed has been paid for and done a literature search on a given item, will that information only be communicated to the individual who hired them, or will it be made more widely available?
A related question: Assuming that the information remains private (as seems to be the most viable business model) will the company attempt to place restrictions on what the clients may do with the information? That is, is the client free to publish it?
Clients are free to publish whatever they like, but we are very strict about patient confidentiality, and do not release any patient information without express written consent.
I like the idea of clients being free to publish anything... but what will you do if they misrepresent what you said, and claim they got the information from you? If could be a honest mistake (omiting part of information that did not seem important to them, but which in fact changes the results critically), oversimplification for sake of popularity ("5 things you should do if you have cancer" for a popular blog), or outright fraud or mental illness. For example someone could use your services and in addition try some homeopatic treatment, and at the end they would publish your advice edited to include the recommendation for homeopathy.
So there should be a rule like: "either publish everything verbatim... or don't mention our name". (I guess you probably already have it, but I say this for the case you don't.)
We won't publish anything, but clients are free to publish whatever they wish to in any manner that they wish.
caveats: they're new; it's hard to do what they're doing; they have to look serious; this is valuable the more it's taken seriously.
They have really wonderful site design/marketing...except that it doesn't give me the impression that they will ever be making the world better for anyone other than their clients. Here's what I'd see as ideal:
On a more profiteering viewpoint, they could offer a report for either $5k for a private report, or $3k for a public report, with a promise to charge $50 for the public report until they reach $5k (or $6k, or an internal number that isn't unreasonable) and then release it.
Most people who are seriously sick tend to get into a pretty idealist...
Shouldn't there be a disclosure of some sorts that MetaMed shares some sponsors with MIRI?
Simple diagnostic tools which may even ignore some data give measurably better outcomes in areas such as deciding whether to put a new admission in a coronary care bed (Green and Mehr 1997).
Better outcomes than what? Typical doctors' diagnostics, I assume?
Shouldn't there be a disclosure of some sorts that MetaMed shares some sponsors with MIRI?
I thought that was obvious by listing Jaan Tallinn as an x-risk funder and Peter Thiel, but yes, you're very correct that this should be explicitly stated on general principles. Will edit.
"Hey, MIRI folks, we're giving you a lot of money, how about you said a couple of nice words about this other company of ours?"
"Hey, our sponsors are funding another company, maybe if we helped promote that company and it ended up doing well, our sponsors would have more money to give us."
""Hey, our sponsors are funding another company, maybe if we helped promote that company our sponsors would be grateful and give us some extra money."
For the sake of humanity, cute kittens, whatever it takes to get past your qualms about this being advertising...
Please promote this immediately to the front page so it can get as much attention as possible.
I'm overall not impressed- looking at their reports, what do they offer that up-to-date (uptodate.com) doesn't? Sure, they advertise at patients, and up to date is aimed at institutions- but most hospitals I'm familiar with (and hence almost any specialist physician) are going to have access already to up-to-date. Also, in general, I'm willing to bet most doctors are in a better position to digest a research report than the average patient.
Sure, its a good idea, but its already being done in a very comprehensive fashion by a company that already has something like 90+% market penetration for academic hospitals. What is metamed's comparative advantage?
That seems a very tricky proposition- for $5k you get a team of medical students and phd students doing a literature search for 24 hours. Without a diagnosis to start with and without an ability to order and receive test results (even if you suggest a test, will the results be back in 24 hours?) my prior would be that diagnosis would be extremely unlikely. WIthout a diagnosis, I'm not even sure how informative such a short literature search can be.
In the case of symptoms-just-started/no diagnosis, doesn't an experienced doctor at a hospital (with all the support staff a hospital implies-. labs,etc) have a pretty high competitive advantage? Apriori, an experienced physician with diagnostic equipment and several days should outperform some medical students with journal access and 24 hours.
Also, this whole thread I find myself shilling for the status-quo, but I should make it clear- hospitals scare the hell out of me. I've done statistical work for internal performance reviews for a large carrier in Southern California and found tons of alarming medical mistakes. I just don't see how Metamed solves any of the actual problems. Most mistakes are of the form transfer orders go t...
UpToDate provides a summary of research based on disease. i.e. for this disease, these treatments are recommended because of study A,B and C and physiological facts D, E and F. There are counter-indications and risks from these treatments because of x,y, and z. Unfortunately, I can't reproduce one of their reports here, but its not just a huge literature dump, its summarized and treatment options are graded.
Looking at the metamed concierge report on Gout (linked to elsewhere in this thread) its formatting appears to be very much like an UpToDate report- the most recent literature is digested and summarized at a decently high level, but it doesn't strike me as better than (or even different from!) the UpToDate recommendations. Given that 90% of academic hospitals already have paid for UpToDate, and honestly in most cases it will be better for the physician to interpret the report, I can't see very much for metamed to bring to the table.
Also worth pointing out- the people who write the summaries for UpToDate are most often researchers in the field of the illness. Near as I can tell from their webpage metamed's researchers are often medical students, or non-medical phd students (the point being that with metamed you are paying for something general called "expertise" and in many cases not actual field-relevant medical expertise).
Only one out of 21 obstetricians could estimate the probability that an unborn child had Down syndrome given a positive test
Say the doctor knows false positive/negative rates of the test, and also the overall probability of Down syndrome, but doesn't know how to combine these into the probability of Down syndrome given a positive test result.
Okay, so to the extent that it's possible, why doesn't someone just tell them the results of the Bayesian updating in advance? I assume a doctor is told the false positive and negative rates of a test. But what matters to the doctor is the probability that the patient has the disorder. So instead of telling a doctor, "Here is the probability that a patient with Down syndrome will have a negative test result," why not just directly say, "When the test is positive, here is the probability of the patient actually having Down syndrome. When the test is negative, here is the probability that the patient has Down syndrome."
Bayes theorem is a general tool that would let doctors manipulate the information they're given into the probabilities that they care about. But am I crazy to think that we could circumvent much of their n...
Am I correct in thinking this is a continuation of the vanished company Personalized Medicine?
What's the story there?
This is a new service and it has to interact with the existing medical system, so they are currently expensive, starting at $5,000 for a research report. (Keeping in mind that a basic report involves a lot of work by people who must be good at math.) If you have a sick friend who can afford it - especially if the regular system is failing them, and they want (or you want) their next step to be more science instead of "alternative medicine" or whatever - please do refer them to MetaMed immediately.
A friend of mine suffers from debilitating effects of fibromyalgia, to the degree that she had to quit her job. She has tried all possible conventional and alternative medicine, with little success. She would certainly be prepared to pay $5000 or more for a near-certain relief, but not for yet another literature search of undetermined benefit. I'm guessing she is not the target audience for MetaMed?
Well, according to their FAQ, they offer a trial service. So your friend would not have to continue to a larger report if the trial seemed to be indicating low benefits of further research.
Can I try MetaMed before committing to a large purchase? Yes. If your case has a larger budget, we can start with a smaller, trial report to ensure quality of service, and confirm that MetaMed is the right choice for you.
And they also offer Financial aid - There is almost no information about this posted, other than that it exists. I guess you would have to call to determine more about how it worked. If your friend did qualify, that would be a substantial boon:
http://metamed.com/financial-aid
Once you have consulted with our medical team, if you need financial aid to help with the cost of your MetaMed service, we will email you an application right away.
And it looks like overall there are at least three tiers of potential research:
http://metamed.com/personalized-research-for-individuals
And here are examples of reports at each tier:
http://metamed.com/static/Meta_Sleep.pdf (Standard)
Those results do not impress me as to the value of their research. There is nothing there that isn't covered by Up-to-Date (http://www.uptodate.com/home) and every hospital I've done stats work for (several, all over the country both community and academic) has provided their physicians with up to date access.
Your best bet (apparently) would be simply asking your physician for the up-to-date report for your diagnosis. If your physician does not have up-to-date access, get a referral to the nearest academic center.
I'm unsure why I've been voted down here- if shminux's friend wants a version of a report similar in quality to what metamed can provide she can ask her physician if the physician has up-to-date (or an other research aggregator) access. If the physician doesn't, its potentially a measure of quality (which can otherwise be hard to judge), and she should get a referral to an academic medical center, which will definitely have something like up-to-date available. This seems to me like decently practical advice for those who have insurance, but don't have the money for metamed. I'm still relatively new so I'm requesting explicit feedback to improve the quality of my posts.
Edited to Add: According to the company, > 90% of American academic hospitals already have subscribed to Up-to-Date, so getting a referral to an academic center has a great chance of getting you to someone with already-paid-for-access to this sort of report.
I am under the impression that IBM's Watson is being tested in a few hospitals for something which seems at least somewhat similiar to Metamed, but I don't know enough about either to really judge well. Sample link to what I am referring to:
Is anyone familiar enough with both Metamed and Watson to help me compare and contrast the support provided by the two of them?
From what I understand, Watson is more supposed to do machine learning and question answering in order to do something like make medical diagnoses based on the literature.
MetaMed tries to evaluate the evidence itself, in order to come up with models for treatment for a patient that are based on good data and an understanding of their personal health.
They both involve reviewing literature, but MetaMed is actually trying to ignore and discard parts of the literature that aren't statistically/logically valid.
According to their site Jaan Tallinn is not the CEO but chairman of the board. Zvi Mowshowitz is the CEO.
Have enough people at MetaMed been influenced sufficiently by (meatspace) LessWrong/think 'similarly enough' to LW rationality that we should precommit to updating by prespecified amountson the effectiveness of LW rationality in response to its successes and failures?
At a first glance, I'm not sure humans can update by prespecified amounts, much less prespecified amounts of the right quantity in this case: something like >95% of all startups fail for various reasons, so even if LW-think could double the standard odds (let's not dicker around with merely increasing effectiveness by 50% or something, let's go all the way to +100%!), you're trying to see the difference between... a 5% success rate and a 10% success rate. One observation just isn't going to count for much here.
it's spam, but it's our spam. upvoted. (I don't mean I work for Meta; I just support the Cause and things/community supporting it)
Upvoted, but I'm a bit confused as to what we're trying to refer to with "spam".
If by spam we mean advertising, yes. Definitely.
If by spam we mean undesirable messaging that lowers the quality of the site, then I would think that this is very much not spam.
Some people (myself included) use "spam" to refer to any kind of advertising in a public setting, e.g. you might preface an email sent out to multiple mailing lists as "sorry for the spam, guys, but..." even if it's a valuable and high-quality email. The connotation, to me, is mildly self-deprecating rather than strictly negative.
Seeing these statistics has got me thinking.
I've checked the undergraduate course requirements as my local university's medical faculty, and there's nothing listed for probability and statistics. I'm considering setting up an appointment with somebody about this, assuming doctors not being able to uncover test results properly is a serious problem.
Would this be worth it, or am I wasting time?
Hanson recently commented on MetaMed on OB, but not here, so might as well quote some of it:
...I wrote this post because I know several of the folks involved, and they asked me to write a post endorsing MetaMed. And I can certainly endorse the general idea of second opinions; the high rate and cost of errors justifies a lot more checking and caution. But on what basis could I recommend MetaMed in particular? Many in the rationalist community think you should trust MetaMed more because they are inside the community, and therefore should be presumed to be mor
This seems like a really good idea. Especially given the impossibility of a single doctor keeping up with all the literature...
Moreover, I rather expect Metamed to be able, ater a while, to suggest profitable research opportunities to people looking to do medical research.
This is a new service and it has to interact with the existing medical system, so they are currently expensive, starting at $5,000 for a research report. (Keeping in mind that a basic report involves a lot of work by people who must be good at math.) If you have a sick friend who can afford it - especially if the regular system is failing them, and they want (or you want) their next step to be more science instead of "alternative medicine" or whatever - please do refer them to MetaMed immediately.
What might it be worth to people to find out that some or all of the usual procedures are so dangerous and/or ineffective as to be not worth doing?
I'd be interested in the linked Begg's paper but it's behind a paywall. Can someone please tell what exactly they had done and how did they obtain all those various p-values?
What are the capital investments that need to be recouped in the early adopter period? Is the price tag based on "It is worth more than this to our target market.", or "It costs this much to do this research, with reasonable amortizing of capital costs."?
In a world where 85% of doctors can't solve simple Bayesian word problems...
In a world where only 20.9% of reported results that a pharmaceutical company tries to investigate for development purposes, fully replicate...
In a world where "p-values" are anything the author wants them to be...
...and where there are all sorts of amazing technologies and techniques which nobody at your hospital has ever heard of...
...there's also MetaMed. Instead of just having “evidence-based medicine” in journals that doctors don't actually read, MetaMed will provide you with actual evidence-based healthcare. Their Chairman and CTO is Jaan Tallinn (cofounder of Skype, major funder of xrisk-related endeavors), one of their major VCs is Peter Thiel (major funder of MIRI), their management includes some names LWers will find familiar, and their researchers know math and stats and in many cases have also read LessWrong. If you have a sufficiently serious problem and can afford their service, MetaMed will (a) put someone on reading the relevant research literature who understands real statistics and can tell whether the paper is trustworthy; and (b) refer you to a cooperative doctor in their network who can carry out the therapies they find.
MetaMed was partially inspired by the case of a woman who had her fingertip chopped off, was told by the hospital that she was screwed, and then read through an awful lot of literature on her own until she found someone working on an advanced regenerative therapy that let her actually grow the fingertip back. The idea behind MetaMed isn't just that they will scour the literature to find how the best experimentally supported treatment differs from the average wisdom - people who regularly read LW will be aware that this is often a pretty large divergence - but that they will also look for this sort of very recent technology that most hospitals won't have heard about.
This is a new service and it has to interact with the existing medical system, so they are currently expensive, starting at $5,000 for a research report. (Keeping in mind that a basic report involves a lot of work by people who must be good at math.) If you have a sick friend who can afford it - especially if the regular system is failing them, and they want (or you want) their next step to be more science instead of "alternative medicine" or whatever - please do refer them to MetaMed immediately. We can’t all have nice things like this someday unless somebody pays for it while it’s still new and expensive. And the regular healthcare system really is bad enough at science (especially in the US, but science is difficult everywhere) that there's no point in condemning anyone to it when they can afford better.
I also got my hands on a copy of MetaMed's standard list of citations that they use to support points to reporters. What follows isn't nearly everything on MetaMed's list, just the items I found most interesting.
90% of preclinical cancer studies could not be replicated:
http://www.nature.com/nature/journal/v483/n7391/full/483531a.html