POCD lasting more than a few weeks is vanishingly uncommon in the young, and the overall mortality rate for anaesthesia is impressively low. The doctor who claimed you were more likely to die in a car crash driving to the hospital than from anaesthesia was exaggerating a little* - but if it's a very long drive to the hospital, say five hours each way, his argument might still stand a chance.

*estimate based only on deaths from anaesthesia. Usually if you're under anaesthesia, lots of other dangerous things are going on at the same time, but deaths from surgical trauma or the condition that necessitated surgery in the first place aren't counted in those numbers.