gjm and Lumifer, thanks for the detailed discussion.

I want to clarify a few points, mainly regarding the context of what I'm writing. My goal was to give an intuition about multiplicity cropping up in different contexts in 400 words, not to explain the details of financial engineering or the difference between t and Z scores.

There are advanced approaches to annualizing Sharpe Ratios but the first method that people are taught is to:

1. Take the average daily return over a number of days, and multiply that by 252 (number of trading days in a year) to get the yearly return.
2. Take the daily standard deviation and multiply by sqrt(252) to get the yearly standard deviation.
3. Subtract the risk-free return from #1 and divide by #2.

Similarly, the basic way that people are taught to test for statistical significance is:

1. Calculate the average measurement.
2. Calculate the standard deviation of the average by dividing the sd of the sample by sqrt(sample size).
3. Subtract the null (usually 0) from #1 and divide by #2.

As a convention, Sharpe Ratios are standardized to a single year and statistics in science (e.g. a drug effectiveness) are standardized to a single average person. If everyone measured daily Sharpe Ratios (instead of yearly) and the effects of drugs on groups of 252 people at once, whether we multiply or divide would switch. But at the core, we're doing the same thing: making a lot of assumptions about how something is distributed and then dividing the "excess" result for a standard unit by the SD of that unit.

I agree that in practice people look at those things very differently. In social psychology you get p 2) and go publish, while in stock-picking Sharpe Ratios rarely get anywhere close to 2, and you compare the ratios directly instead of thinking of them as p-values. Still, both measurements are equally affected by testing multiple hypotheses and reporting the best one. If someone tells me of a stock picking strategy (17th lag!) that has a Sharpe Ratio of 0.4 (as compared to the S&P's 0.25) but they tried 20 strategies to get there, it's worth as much as green jelly beans. That's all I was trying to get at in the first half of the post, and I don't think anyone disagrees on this point.

Heh, nope. Finance people (other than marketers) are very interested in empirical truth because for them the match between the map and the territory directly translates into money. Hope is not a virtue in finance.

And that's exactly what I was trying to get at in the second half of the post.