for most people this will not happen

That's a remarkably low bar.

A great deal of things will not happen "for most people". Getting academic tenure, for example. Or having a net wealth of $1m. Or having travelled to the Galapagos Islands. Etc., etc.

First, OP mentions stock market

Yes, but that's the basic uninformed default choice when people talk about financial markets. It's like "What do you think about hamburgers? Oh, I think McDonalds is really yucky". Um, there's more than that.

If you look at what's available for an individual American investor with, say, $5-10K to invest, she can invest in stocks or bonds or commodities (agricultural or metals or oil or precious metals or...) or currencies or spreads or derivatives -- and if you start looking at getting exposure through ETFs, you can invest into pretty much anything.

The focus on the stock market is pretty much a remnant from days long past.

A lone but very smart and rational thinking programmer isn't likely to win.

I don't know. It depends on how smart and skilled he is.

He might also join forces with some smart friends. Become, y'know, one of those teams of "top tier mathematicians and computer engineers" who eat the lunch of plain-vanilla investors. But wait, if the markets are truly efficient, what are these top-tier people doing in there anyway? :-/

Why would you choose to make that the playground for you to test your predictions skills?

Because the outcomes are direct and unambiguous. Because some people like challenges. Because it's a way to become rich quickly.

Strategies that are ignored by the truly big players are picked up by the countless mutual fund managers

Mutual fund managers are very restricted in what they can do. Besides outright constraints (for example, they can't go short) they are slaves to their benchmarks.

But strong stat arb might as well be just as good as riskless if you truly have an edge.

Oh, no. "Riskless" and "I think it's as good as riskless" are very, very different things.

a "risk-free" arbitrage by simply applying your alpha producing strategy and simultaneously shorting S&P etf's to create a stat arbitrage.

That doesn't get you anywhere near "riskless". That just makes you hedged with respect to the market, hopefully beta-hedged and not just dollar-hedged.

Strategies by nature are ephemeral

True, but people show a very consistent ability to come up with new ones when old ones die.

In any case, no one is arguing that you can find a trade or a strategy and then milk it forever. You only need to find a strategy that will work for long enough for you to make serious money off it. Rinse and repeat, if you can. If you can't, you still have the money from a successful run.