If the problem is computationally difficult, then it is a mistake to not count the value of computation.
Suppose you believe that a reasonably smart person with some spare time can make money on the stock market. There is still a question of how much money, compared to the amount they could earn doing other things. If we pick some reasonable value for intelligent, market understanding thought, then we can define a new value which is - the value of the time it took to find.
The value of thought time, compute time and data are all constant. The returns are linear until you get to the point where you disrupt the market, destroying the pattern. Thus is much harder to get if the amount of money you have to invest is small. (Especially if you don't bet the farm.)
There are different reasons why alpha exists and persists. One misconception that seems to be common is that alpha could only persist because of secrecy. Now that kind probably exists, but finding and exploiting it is catching lightning in a bottle. It's awesome if you can do it, but it's probably not worth the effort. There are easier games.
Some types of alpha do indeed require a lot of compute to find, but you can speed that up by renting compute in the cloud (which does cost some money). And you can search more efficiently by having educated guesses about where to look, which also seems like something rationalists might be good at. And yes, how much you can take advantage of that depends on your capital. Seen from the other side, this is one of the advantages of trading though. You can earn faster by saving more money. It scales in a way a normal day job doesn't, which may only give you small raises that sometimes don't even keep up with inflation.
But you can do this kind of trading more efficiently by cooperating with other agents. You can share information. The deal works like this: you tell the others what your edge is, they reciprocate and tell you about theirs. This is a cost to you. The information wasn't free, and sharing it diminishes its value, because alphas diminish the more they're traded. However, you didn't have the capital to max it out yourself anyway. Your friends probably don't either, so all of you can trade it for a while. If you have a dozen trader friends and you each had one edge, now you all have a dozen plus one edges, with the benefits of diversification. Thirteen edges for the price of one. Secrecy isn't always the best move.
(This is also one of my motivations for writing this sequence. More trader friends means more edges for me!)
Other edges persist because people, for one reason or another, find exploiting them distasteful. E.g. a skewed risk profile. In some cases, it may simply be constrained by capital. It could be a very good deal for a small trader, but a large trader with different opportunities wouldn't bother, because they can't trade it with sufficient volume to be worth the effort.
Because the markets are so efficient, the market doesn't punish you much for being wrong
Could you explain this cause-effect a bit more? My intuition says if I make the wrong choice where the vast majority is making the right choice, my losses will quickly get snapped up into everyone else's gains
Turn the question around: assume your goal is to lose money. If a market was 100% efficient, the price moves would be 100% unpredictable, i.e. random. You're just as likely to make money as to lose it trading such a market. How could you lose money if you tried? Think about it.
I can only think of a few ways.
The first, obvious, way is to trade a lot. You lose commissions and spread each time you trade. Even if some of your trades happen to make a lot of money, you can lose more by overtrading. How quickly this happens depends on the size of the spread and commissions, and the frequency of your trades. You lose more slowly this way if you trade less often. Hence, DON'T PANIC.
The second way is leverage. If 100% of your portfolio value is within the expected move of your trade, then you stand to make a lot of money on the upside, but you lose everything on the downside. Even if you happen to start in a winning streak, you are almost certain to eventually lose it all. But 100% is not required. As long as you're over the Kelly fraction, you'll lose the entire bankroll eventually. Hence, Don't Bet the Farm.
The third way is more subtle. Even assuming an efficient market, the buy-and-hold strategy can work over time due to the risk premium. The random walk has a negative skew with a positive drift. You'll hit a few home runs when the market crashes, but, over the long term, a short-and-hold strategy will lose money. Hence, Don't Fight the Wave.
I can think of numerous individual strategies for losing money, but they all seem to fall into one of these three categories, assuming an efficient market.
There is a fourth way to lose money trading, but it breaks one of our assumptions: trade in an inefficient market. Alpha is difficult to find. You're probably no more likely to stumble upon a negative alpha trade than a positive one. Find some alpha, and then make the opposite bet.
I thought of a few more. Taxes, fees, and penalties can cost you. Be especially careful about mutual funds, which can charge outrageous amounts in an attempt to keep you locked in. One can avoid a lot of taxes by using retirement accounts, but you really can't take the other side of these deals.
Inflation is another big one. You're not technically losing anything in nominal terms, but your buying power does shrink over time. The Fed's 2% target is not such a big deal for one making 20%, but sometimes inflation is much higher.
Bear market risk is fairly easy to understand: a rapid selloff decreases the value of stocks you own; in which case one is better off holding cash or commodities. That's left-tail risk. OTM puts are left-tail insurance.
Right-tail risk is less intuitive. After all, if your stocks rapidly appreciate, isn't that a good thing? But a period of high inflation can also inflate stock prices, although the relationship is complicated (value stocks tend to do better, while growth stocks may be hurt by the economic effects of inflation by more than their prices inflate). Inflation is a Red Queen race: you have to run (in dollar terms) just to hold your position (in buying power terms). In a period of higher inflation, one has to run faster to keep up. OTM calls are right-tail insurance (there is also a sense in which they're equivalent to a married put). Commodities (but not cash) can also be helpful here.
Even without high inflation, missing out on the right tail can mean being left behind compared to a non-dollar benchmark, like passive index investing. Hence the buy-and-hold adage about time in the market rather than timing the market. However, cutting off both tails would have done similarly well, at least historically. You can theoretically do that with a costless options collar, i.e., sell a covered call to fund an OTM (married) put, although IV skewness across strikes makes that less obviously a win as the downside has to be further OTM. Using (e.g.) VIX calls as insurance may be more efficient, but it's also more complicated.
Thanks, this is very valuable. I'll have to think about this some more; I don't think I've internalized it enough yet:
If a market was 100% efficient, the price moves would be 100% unpredictable
Also, are you sure your definition of beta matches Wikipedia's? I'm not seeing how they can be the same thing
I am using the term loosely. Alpha and Beta come from the CAPM, where they have a very specific meaning. Perhaps I should have said "risk premium" instead of "beta". That's closer to what I mean. And by "alpha" I mean what's left after subtracting out the risk premium and luck: the skill of the trader.
Part 5 of the Inefficient Markets sequence.
[I am not your financial advisor! For educational purposes only. Read part 1 first.]
You, dear reader, are probably not average. I expect a certain selection bias for readers of this blog. You know how to use a spreadsheet. You can probably write code. You can apply mathematics to real life. You're aware of your own cognitive biases, to some degree. You understand why that's important.
Making good decisions under uncertainty is the very essence of trading. As I said in part 1, rationalists are already halfway to quant. Do you have any idea how much money is moving through those markets? There's gold in those hills.
So why aintcha rich?
Different readers may answer this differently, but it may go something like this.
Because you haven't been trading. Why? Because you don't know how. Why? Because you never studied how. Why? Maybe you thought you had better things to do. Maybe you didn't think it was worth the effort. Maybe you didn't realize how much we all need more money.
Maybe because you don't believe you can succeed.
Why?
Trading Is Dangerous
Most retail traders fail to beat the index. Monkeys throwing darts at newspapers do a better job of picking stocks than the pundits. When you trade with borrowed money, you can lose more than you invested. For some people, trading is a ruinous gambling addiction.
Successful Trading Is Impossible
The Efficient Market Hypothesis means you can't exploit the markets They're random. There's no such thing as alpha--reward for skill instead of risk. Successful alpha traders are an illusion. They're just lucky. People also win the lottery, but that doesn't change the fact that it's a bad investment.
There's still beta--reward for taking risk. But the only way to get a higher return is to take on more risk.
Not That Strong
The strong form of the EMH, the idea that markets are perfect, that even private information is reflected in the stock price, is too strong. The strong form is simply implausible given insider trading laws. That these laws are a good idea is a matter of some debate among economists. Nevertheless, the laws are there.
But you don't have insider information, and probably couldn't trade it if you did.
The semi-strong form, that allows for the possibility of private information not reflected in the price, might as well be strong for you.
Weak Is Strong Enough?
The Weak EMH means that there's no such thing as technical analysis: all past price, volume, and earnings information is already priced in. Maybe there's some kind of fundamental analysis that could work. But the traditional fundamental methods don't seem that effective anymore. You can't reliably outperform the market, at least in the short term.
Maybe you could get an edge if you hire a team of investigators, to find the "public" information that isn't very accessible. But it doesn't seem feasible for an individual.
There Is No Alpha Left for Me
Maybe you don't accept the EMH, but think you have to be some kind of trading genius who has dedicated his life to a specialty in finance and some kind of inside knowledge to have a chance. That's your competition. You don't really expect to beat these guys in a zero sum game, do you? You can't win.
The only winning move is not to play.
Trading costs you spread and commission. But if alpha can't be had, why bother? Just buy the index and forget about it for ten years? "I'm not a trader, I'm an investor," right?
Sour grapes!
You're not smarter than everybody. But you're a rationalist. You're smart enough.
Trading Is not That Hard
A young mathematician began his lecture, scrawling equations over several blackboards, his esteemed colleagues looking on with reserved interest. He droned on for several minutes, until stopping mid-sentence, mid-scratch, and with a look of intense concentration, he erased and rewrote a few symbols. And again. Chin in hand, he regarded his work, for a time.
"Huh."
Frantically, he flipped through his paper notes, adjusting his spectacles to focus on the page, which he turned, and turned again. "It's not right!" Finally at his wit's end, he stormed out of the room in consternation.
Fifteen minutes later, bursting into the room, with a spring in his step, and a snap of his fingers, he brightly interrupted his colleagues' quiet conversation with "It's trivial!"
It's only hard until you get it. Then it's trivial. Sometimes.
Let's not pretend for a moment that successful trading is a trivial endeavor. But difficulty isn't a property of the task alone. It's a relation between the nature of the task and the doer's skill.
This should not be physically taxing. It's not like you're digging ditches. You're clicking buttons and typing things into a computer. And most importantly, it doesn't have to take 40 hours a week.
Let's not lose sight of our instrumental goal here: decouple time from income. As much as you can. If you've found an easier way (for you), then by all means, do that. But trading seems like an accessible small business for the rationalist type.
Trading Is Safer Than You Think
Because the markets are so efficient, the market doesn't punish you much for being wrong, as long as you take reasonable precautions. You're unlikely to actually find much negative alpha you're not looking for. An efficient market will be fair to you.
The Three Laws are the reasonable precautions. They give you a safe framework to attempt alpha trades from. You must, of course, have a high enough level of confidence in your analysis, but you are never going to be certain of your edges, even when you find them. So diversify and trade a portfolio of alpha strategies. If some of them fail, so what? They won't hurt you, because you--
DON'T PANIC, which ensures that you trade systematically, and not so frequently that you lose too much money on spread and commissions. It's also a warning about giving up too easily. Drawdowns are normal and expected even when your system works.
Don't Bet the Farm which keeps you from losing the fair games. Wins and losses will tend to cancel out if you have no alpha, because the market is efficient enough.
Don't Fight the Wave which keeps your portfolio on the right side of beta. It compensates you for the inevitable losses from spread and commissions, and gives a tailwind to your alpha trades that might otherwise be too small of an edge to exploit.
Alpha Exists
You get a spam email advertising thinly traded penny stock that's about to go up. Is this a good deal, or should you be worried?
The EMH is the spherical cow of finance. It can be approximately true enough to be a useful assumption in liquid markets.
But understand that this is a shortcut to make some equations tractable. It can't be literally true, even in its weakest form, because technical alpha exists.
Alpha is any exploitable non-randomness in market behavior. Simple statistical tests of randomness prove that while the markets are extremely noisy, they aren't completely random. If you're already familiar with any of these tests, I would encourage you to download some price data and try them. You might be surprised by what you find. Can you exploit these patterns? Now you're starting to think like a quant.
That spam is part of a pump-and-dump scheme. You are right to fear this, but if you do, you're admitting that that market is not efficient. If you take the opposite side, and fade before the crash, that's an alpha trade!
There are degrees of market efficiency. But scams aside, even liquid markets bubble and crash. The only difference is scale. The effects of bias and ignorance are more pronounced for thinly-traded assets, but liquid assets aren't perfectly efficient either.
Alphas are unstable. They diminish over time, eventually becoming unprofitable. You do have to keep finding new ones. But it can take longer than you might expect for an edge to dull. Alphas don't persist simply because of secrecy. There are cases published in the academic literature that still show up in recent price data. And that's exactly the kind of alpha you can be most confident of.
Perfect market efficiency is computationally intractable, especially when you consider relations between assets. Pairs trading is a notable example of this kind of effect. There are many more pairs of assets than there are assets. And this is even more true of baskets of more than two. There are too many combinations for anyone to calculate. We are not going to run out of alpha.
There are many other examples of alpha. The rest of the sequence will be about how we go about finding and exploiting some of them.
Part 6 is next.