I think focusing on the chance of cancer is a red herring. Human brains being what they are, even if you notice you're privileging the cancer hypothesis, you're still going to worry. Sure, do the math, but save that result for another calculation.
How much utility are you likely to derive from $1000? How badly does suspecting cancer disturb your peace of mind? And how much is your ability to derive utility going to be reduced when you are without peace of mind?
We could do the math, but it seems clear to me you know something about the shape of these equations: you opted for the biopsy.
Today, my dentist found a possible oral cancer.
I'm 31, a non-smoker, in good health. I know the research showing that doctors ignore base rates and overestimate your chances of cancer. (I asked the doctor the base rate, he didn't know.) I know that we grossly overprescribe biopsies and surgeries, when it would be better to just wait and see. But I'm having it removed and biopsied on Friday, even though I don't have dental insurance and it's costing me $1,000 of my own money.
Why?
I thought this would be an interesting case study: Introspectively, what's going on to make me ignore my rationalist training, ignore the external data, and choose what I know is probably the less optimal path?
My first thought is embarrassment: If I do nothing, and it turns out I have cancer, will my support network roll their eyes and blame me for not being more aggressive? My feeling is yes, that even though they wouldn't do it to my face, they would secretly blame me, and become less available.
My second thought is fear of the unknown: I roughly know what the biopsy will entail. It's a light anesthesia, a few stitches, and 1-2 days of recovery. No big deal. And $1,000 isn't tiny, but it's not a big deal for me, either. In contrast, what happens if I don't do the biopsy? Huge, scary unknown. And, even if I know that I only have a 0.01% chance of having cancer (to guess a number), I also know my emotional mind is bad at math, and I'll have great difficulty controlling its worry. And so, it's rational to buy some level of anti-worry insurance -- I don't know what the rational value of that anti-worry insurance is, and I don't know if that value exceeds $1,000, but clearly, anti-worry insurance has some positive value, and probably a fairly high value.
There are other considerations. I need some wisdom teeth removed, and we're doing them at the same time, so adding the biopsy doesn't affect the recovery time. And I have a 6-week trip to Australia coming up, and I'd hate to have problems while I'm traveling. But mostly, I think it's embarrassment and worry.
By the way, this is a personal matter than I'm choosing to share with you. Honest advice about how to handle it, particularly from people who've faced similar decisions, is welcome and appreciated. Flames about how I'm ignoring research are not. Thanks for understanding.
Update: I talked with a friend who does research in medical decision making. She explained that, for the specific population a doctor serves, he's usually fairly accurate in his estimates of how prevalent a disease is. She also encouraged me that I'm right to get the biopsy. I feel much more relaxed, and I realized that I was feeling guilty about being irrational. I'm sure there's some meta-lesson in there that I'll figure out someday.