More precisely, the question is whether or not the right reference class is one in which cancer tests are worth while. The headaches would have to be very unusually severe to get enough evidence.

Also, a doctor can ask for details about the headache before prescribing the expensive tests.

It was never mentioned whether or not the doctor asked for details. It's also possible that none of those reference classes are worth looking into, and she'd need headaches and something else.

The argument by majority fallacy means arguing that something is true because many people believe it. In the example of the headaches, the argument was that it was likely true because it is true for most people.

What you would want your doctor to do is take the action that maximizes your expected utility, E[U(action)]. Let's simplify a bit, and say that action can be either "do nothing" or "find cause". Then the utilities could be something like:

P(not sick) = 0.99 (most people have nothing to worry about)
U(not sick, do nothing) = 0
U(not sick, find cure) = -1 (unnecessary tests, drugs, worry)
U(sick, do nothing) = -10 (possibly more headaches, or something worse)
U(sick, find cure) = -1 (still need tests, drugs, etc.)

Then:

E[U(do nothing)] = P(not sick) * U(not sick, do nothing) + P(sick) * U(sick, do nothing) = -0.1
E[U(find cure)] = P(not sick) * U(not sick, find cure) + P(sick) * U(sick, find cure) = -1

So with these numbers I just made up, it is better for the doctor to tell you that there is likely nothing to worry about. And you can be pretty sure that in real life, people have done this calculation. Of course in real life there are many more possible actions, such as waiting for a week to see if the headaches go away, which they will likely do if there was nothing wrong. And that is what a doctor will actually tell you to do.

I'm pretty sure that, in this case, the probability is smaller than the disutility is large. Getting tested for cancer doesn't come cheap.

Doctors get taught to practice evidence-based medicine. There's a lack of clinical trials that show that you can increase life span by routinically giving people who suffer from headaches brain scans.

If I understand the argument right, then doctors are basically irrational because the favor empirical results from trials over trying to think through the problem on a intellectual level?

What's wrong with “you most likely have nothing to worry about, but I suggest doing this exam the off-chance that you do”?

MONNAY.

You've got to multiply the probability by the disutility, and the result can be large enough to worry about even if the probability is small.

The question is, whose utility?

There's also the problem of false positives. Treatments for rare diseases are often expensive and/or carry serious side effects.