Related: Circular Altruism
One thing that many people misunderstand is the concept of personal versus societal safety. These concepts are often conflated despite the appropriate mindsets being quite different.
Simply put, personal safety is personal.
In other words, the appropriate actions to take for personal safety are whichever actions reduce your chance of being injured or killed within reasonable cost boundaries. These actions are largely based on situational factors because the elements of risk that two given people experience may be wildly disparate.
For instance, if you are currently a young computer programmer living in a typical American city, you may want to look at eating better, driving your car less often, and giving up unhealthy habits like smoking. However, if you are currently an infantryman about to deploy to Afghanistan, you may want to look at improving your reaction time, training your situational awareness, and practicing rifle shooting under stressful conditions.
One common mistake is to attempt to preserve personal safety for extreme circumstances such as nuclear wars. Some individuals invest sizeable amounts of money into fallout shelters, years worth of emergency supplies, etc.
While it is certainly true that a nuclear war would kill or severely disrupt you if it occurred, this is not necessarily a fully convincing argument in favor of building a fallout shelter. One has to consider the cost of building a fallout shelter, the chance that your fallout shelter will actually save you in the event of a nuclear war, and the odds of a nuclear war actually occurring.
Further, one must consider the quality of life reduction that one would likely experience in a post-nuclear war world. It's also important to remember that, in the long run, your survival is contingent on access to medicine and scientific progress. Future medical advances may even extend your lifespan very dramatically, and potentially provide very large amounts of utility. Unfortunately, full-scale nuclear war is very likely to impair medicine and science for quite some time, perhaps permanently.
Thus even if your fallout shelter succeeds, you will likely live a shorter and less pleasant life than you would otherwise. In the end, building a fallout shelter looks like an unwise investment unless you are extremely confident that a nuclear war will occur shortly-- and if you are, I want to see your data!
When taking personal precautionary measures, worrying about such catastrophes is generally silly, especially given the risks we all take on a regular basis-- risks that, in most cases, are much easier to avoid than nuclear wars. Societal disasters are generally extremely expensive for the individual to protect against, and carry a large amount of disutility even if protections succeed.
To make matters worse, if there's a nuclear war tomorrow and your house is hit directly, you'll be just as dead as if you fall off your bike and break your neck. Dying in a more dramatic fashion does not, generally speaking, produce more disutility than dying in a mundane fashion does. In other words, when optimizing for personal safety, focus on accidents, not nuclear wars; buy a bike helmet, not a fallout shelter.
The flip side to this, of course, is that if there is a full-scale nuclear war, hundreds of millions-- if not billions-- of people will die and society will be permanently disrupted. If you die in a bike accident tomorrow, perhaps a half dozen people will be killed at most. So when we focus on non-selfish actions, the big picture is far, far, far more important. If you can reduce the odds of a nuclear war by one one-thousandth of one percent, more lives will be saved on average than if you can prevent hundreds of fatal accidents.
When optimizing for overall safety, focus on the biggest possible threats that you can have an impact on. In other words, when dealing with societal-level risks, your projected impact will be much higher if you try to focus on protecting society instead of protecting yourself.
In the end, building fallout shelters is probably silly, but attempting to reduce the risk of nuclear war sure as hell isn't. And if you do end up worrying about whether a nuclear war is about to happen, remember that if you can reduce the risk of said war-- which might be as easy as making a movie-- your actions will have a much, much greater overall impact than building a shelter ever could.
First off, regarding your hypothetical, it would be no contest. Replacing earth with a box of ice cream would have about the same decision time. You could frame it as a more active choice, akin to the trolley problem - have me convince Galactus to change course towards Earth - I wouldn't mind.
Now where you go wrong is assuming that somehow implies that I do not value the lives of my fellow human beings, or of mankind overall. On the contrary, I am quite concerned about x-risk, and I would be too if there were no family to be affected. It is just not the priority goal.
Consider you had a choice between the life of a non-human primate, xor that of a human. Just because you (hypothetically) quickly decide to save the human does not mean you do not value the non-human primate, or that without another high-priority preference involved, you would not invest a lot into saving that animal. Do you see the point?
No, why would you think that? I do share that value, and I obviously would as a derived value even if I only did care about my family (repercussions). But even without such considerations, I'd care about it. I'd just accept no tradeoff whatsoever compromising between other humans and "my" humans. What's to defend about that? As I wrote elsewhere, it's not a "bias" in any sense of the word as it's used around here.
Lastly, again with that curious choice of calling preferences "irrational". As if we could just argue towards which preferences to rationally choose. Which ice cream flavor we really should enjoy the most.
How about this? I realize a lot of the points stretch credulity, but I think you should be able to imagine the situation.
Your family member requires a kidney transplant or they will die in 6 months. With the transplant, you can expect they will live an average of 10 additional days. Normal channels of obtaining one have completely failed. By some happenstance, you know of a 25-year-old pretty average-seeming woman who is a signed-up donor (you are not personally acquainted with her), and happen to know that if she dies, your family member will receive the transplant. Do you kill her and make it look like an accident in order to get the transplant, given that you know you would definitely for sure get away with it?