Jacobian comments on Don't You Care If It Works? - Part 1 - Less Wrong

4 Post author: Jacobian 29 July 2015 02:32PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (58)

You are viewing a single comment's thread. Show more comments above.

Comment author: 2irons 29 July 2015 03:41:35PM *  6 points [-]

"Does this change your confidence in Bob managing your retirement investments?" - if he never held himself out as a quant based investor or using a method reliant on much analytical or quantitative research I wouldn't worry about it.

Maybe he's been good at choosing ETF's because he's great at listening to investors and trader chat - can feel which arguments are about to dominate the market and allocates capital accordingly. Maybe he sits by a high performing proprietory trading team in his bank and you're piggy backing off of all their trades at a fraction of the fee. As a fund manager I know several other managers who would have no hope of following most of the articles on this website, misunderstand probability at basic levels (this has been teased out by in depth conversations on things like card counting - where they are high conviction yet wrong) but yet who I'd still have to concede are likely to continue outperforming me in the market because they are great at the parts that count.

I think this is put best by Nassim Taleb in Anti-Fragile:

“In one of the rare noncharlatanic books in finance, descriptively called What I Learned Losing A Million Dollars, the protagonist makes a big discovery. He remarks that a fellow called Joe Siegel, the most active trader in a commodity called “green lumber” actually thought that it was lumber painted green (rather than freshly cut lumber, called green because it had not been dried). And he made a living, even a fortune trading the stuff! Meanwhile the narrator was into theories of what caused the price of commodities to move and went bust.

The fact is that predicting the orderflow in lumber and the price dynamics narrative had little to do with these details —not the same ting. Floor traders are selected in the most nonnarrative manner, just by evolution in the sense that nice arguments don’t make much difference.”

Perhaps I'm being a bit harsh focusing on an analogy but I think there might be a wider point. Producing the right or wrong answers in one domain isn't necessarily a useful predictor of someone's ability to produce the right or wrong answer in another - even when they are highly connected.

Comment author: Jacobian 29 July 2015 08:29:23PM 0 points [-]

Upvoted for criticizing the actual point :)

I agree that Bob is probably a weak analogy because who knows how stock-pickers actually pick stocks? I hope I didn't construct a reverse intuition pump. Still, Eliezer's job is either a research mathematician or a philosopher of epistemology, depending on what matters to you. Both those jobs depend quite a bit on getting the rules of probability right. I think disagreeing with Eliezer on the rules of probability isn't something one can ignore.

You also hit an important point with the question: is Bob a quant researcher? Or more specifically, what's his epistemology in each domain? A few years ago I met Robert Aumann, everyone's favorite theist-mathematician. It's clear to me that Aumann has separate epistemologies for claims about God, Israeli politics and game theory. I read his book about games with incomplete information, he actually goes to the trouble of writing out every single proof, it never says "God told me this is the upper bound". If Aumann tells me "I've proven with rigorous math than an AI boxing system is possible", I sit up and pay attention. If he tells me "AIs don't have souls so humans will always be superior" I won't take that argument seriously.

Eliezer claims and seems to have a single epistemology (EWOR) that he applies to all domains. Because of that, his accuracy in one domain directly reflects on the dependability of EWOR, and thus on his accuracy in other domains.