Related: Circular Altruism
One thing that many people misunderstand is the concept of personal versus societal safety. These concepts are often conflated despite the appropriate mindsets being quite different.
Simply put, personal safety is personal.
In other words, the appropriate actions to take for personal safety are whichever actions reduce your chance of being injured or killed within reasonable cost boundaries. These actions are largely based on situational factors because the elements of risk that two given people experience may be wildly disparate.
For instance, if you are currently a young computer programmer living in a typical American city, you may want to look at eating better, driving your car less often, and giving up unhealthy habits like smoking. However, if you are currently an infantryman about to deploy to Afghanistan, you may want to look at improving your reaction time, training your situational awareness, and practicing rifle shooting under stressful conditions.
One common mistake is to attempt to preserve personal safety for extreme circumstances such as nuclear wars. Some individuals invest sizeable amounts of money into fallout shelters, years worth of emergency supplies, etc.
While it is certainly true that a nuclear war would kill or severely disrupt you if it occurred, this is not necessarily a fully convincing argument in favor of building a fallout shelter. One has to consider the cost of building a fallout shelter, the chance that your fallout shelter will actually save you in the event of a nuclear war, and the odds of a nuclear war actually occurring.
Further, one must consider the quality of life reduction that one would likely experience in a post-nuclear war world. It's also important to remember that, in the long run, your survival is contingent on access to medicine and scientific progress. Future medical advances may even extend your lifespan very dramatically, and potentially provide very large amounts of utility. Unfortunately, full-scale nuclear war is very likely to impair medicine and science for quite some time, perhaps permanently.
Thus even if your fallout shelter succeeds, you will likely live a shorter and less pleasant life than you would otherwise. In the end, building a fallout shelter looks like an unwise investment unless you are extremely confident that a nuclear war will occur shortly-- and if you are, I want to see your data!
When taking personal precautionary measures, worrying about such catastrophes is generally silly, especially given the risks we all take on a regular basis-- risks that, in most cases, are much easier to avoid than nuclear wars. Societal disasters are generally extremely expensive for the individual to protect against, and carry a large amount of disutility even if protections succeed.
To make matters worse, if there's a nuclear war tomorrow and your house is hit directly, you'll be just as dead as if you fall off your bike and break your neck. Dying in a more dramatic fashion does not, generally speaking, produce more disutility than dying in a mundane fashion does. In other words, when optimizing for personal safety, focus on accidents, not nuclear wars; buy a bike helmet, not a fallout shelter.
The flip side to this, of course, is that if there is a full-scale nuclear war, hundreds of millions-- if not billions-- of people will die and society will be permanently disrupted. If you die in a bike accident tomorrow, perhaps a half dozen people will be killed at most. So when we focus on non-selfish actions, the big picture is far, far, far more important. If you can reduce the odds of a nuclear war by one one-thousandth of one percent, more lives will be saved on average than if you can prevent hundreds of fatal accidents.
When optimizing for overall safety, focus on the biggest possible threats that you can have an impact on. In other words, when dealing with societal-level risks, your projected impact will be much higher if you try to focus on protecting society instead of protecting yourself.
In the end, building fallout shelters is probably silly, but attempting to reduce the risk of nuclear war sure as hell isn't. And if you do end up worrying about whether a nuclear war is about to happen, remember that if you can reduce the risk of said war-- which might be as easy as making a movie-- your actions will have a much, much greater overall impact than building a shelter ever could.
Well, but there's also the issue with sums being at all times partial. The low probability high impact scenarios are inherently problematic because very huge number of such scenarios can be constructed (that's where their low probability comes from), and ultimately, your action will be dependent not on utility but on which types of scenarios you are more likely to construct or encounter.
There's also the issue with predictability of the actions. E.g. you can, with a carefully placed flap of butterfly wings, save or kill enormous number of people, but it all balances out - if you are equally able to construct arguments in favour, or against the flap. It's easy for butterfly but it is not so easy for other actions such as donating. Whereas there are clearly possible scenarios (accidental nuclear nuclear exchange) that you can save yourself from with your fallout shelter, and it does not balance out.
Ultimately, it is all up to ability to predict what happens. You can't really predict what happens out of giving money for someone to prevent robot apocalypse. Maybe they'll produce useful insights. Maybe the reason they are so concerned is that their thinking about artificial intelligence is inside a box full of particularly dangerous AIs, and that's where they do all their research, and this actually increases risk or creates even worse scenarios (AIs that torture everyone). Maybe they are promoting notion of the risk. Maybe Frankenstein and Terminator already saturated that. Maybe they look bad or act annoying (non credentiated people intruding into highly technical fields tend to have such effect, especially on cultures that hold scholarship and testing in high regard - e.g. Asians, former Soviet Union, Europe even) and discredit the concerns, making important research harder to publish. You can't evaluate all of that, nor can you produce representative and sufficiently large sample of the concerns, so the expected utility is exactly zero (minus the predictable consequences of you having less money to spend on any future deals). The fallout shelter on the other hand is not exactly zero, it may not be the best idea but you have a clear world model of it not cancelling out.