I refuse to sacrifice my life to protect billionaires who would not do the same for me.

Elon Musk has risked his entire fortune for you. "In my case, I think these things are important... I need to do it, I promised people I would do it, but I'm not doing it because this is the most fun way to live."

The world's wealthiest people (the "ownership class") is increasingly made up of scientists and engineers:

If you work your way down the Forbes 400 making an x next to the name of each person with an MBA, you'll learn something important about business school. After Warren Buffett, you don't hit another MBA till number 22, Phil Knight, the CEO of Nike. There are only 5 MBAs in the top 50. What you notice in the Forbes 400 are a lot of people with technical backgrounds. Bill Gates, Steve Jobs, Larry Ellison, Michael Dell, Jeff Bezos, Gordon Moore. The rulers of the technology business tend to come from technology, not business.

- Paul Graham

Paul has written 1, 2, 3, 4, 5 essays that touch on the topic of why cooperators tend to get rich in Silicon Valley rather than defectors. The Silicon Valley elite is giving their money away significantly earlier in life than previous generations of wealthy people, and there are indicators that they care more about having their philanthropic dollars actually do good--here's tech billionaire Sean Parker on his giving philosophy. (Not to say that other wealthy people are especially lacking in their philanthropy--check out the Giving Pledge signatories.)

Scientists get less sex than criminals.

High IQ people, regardless of gender, have less sex. But it's hard to tease out exactly why. I lean towards Paul Graham's explanation--highly intelligent people tend to be interested in things other than sex, whereas average people structure large amounts of their lives around it (for example, it's typical for every Friday and Saturday evening to be spent drinking carcinogens and searching for sexual partners). More evidence for this hypothesis: Intelligent people seem to be taller and better looking on average. And intelligent friends of mine who have chosen to optimize for having more sexual partners have done well, especially if they're willing to date down in intelligence (to avoid the problem that highly intelligent women are outnumbered by highly intelligent men and also relatively uninterested in sex) and live in an area with a favorable gender ratio. If you want more sexual partners, a good first step is to start working out--it will give you a masculine physique, help you live longer, improve sleep, improve immune system, improve willpower, etc. Once you've spent some time optimizing to increase your number of sexual partners, you'll likely feel less insecure about who your girlfriends have slept with. (And once you've conquered your insecurities, you can work on cool stuff like decreasing existential risk.)

(BTW note that there are more women graduating college nowadays than men, at least in the US, so being educated gives you a leg up.)

all of my ex-girlfriends had sex with someone who doesn't share my values

You're looking at a small number of data points. Psychological research, insofar as it relates to this topic, is more mixed. Research also seems to indicate that having lots of sexual partners is associated with decreased happiness. Those dominant "defector" types are often rejected by women for longer-term relationships, which sucks a lot more than you would think (speaking from personal experience as someone with a dominant/masculinized facial appearance).

It's illegal to punish people who always defect in prisoner's dilemmas.

If there's a particular sort of defection you are concerned about, you can work to change society in order to disincentivize it. This probably isn't the best example, but I've always wondered why we don't punish rapists (and maybe other criminals) with castration. It seems like something that both the far left and the far right could get behind--the far left is full of feminists who think rapists are unadulterated evil, and the far right can appreciate the eugenic benefits of sterilizing criminals. It's cheaper and more humane than locking someone in a hellish prison cell for years on end. It helps solve the root problem, given testosterone's role in facilitating aggression. And it sends the right message to other folks in society. "Here's a man who defected against the rest of us. He speaks in a high voice now because we literally chopped his balls off. He tried to rape a woman, but now he will never have sex again." I think castrated criminals who lived in lower class communities would inevitably get bullied and made fun of, which seems like exactly what we want to happen (as long as someone is going to get bullied and made fun of in lower class communities, which seems inevitable).