do what gwern does

Or do the complete opposite.

The impression I get of gwern is that he reads widely, thinks creatively, and experiments frequently, so he is constantly confronted with hypotheses that he has encountered or has generated. His use of statistics is generally confirmatory, in that he's using data to filter out unjustified hypotheses so he can further research or explore or theorize about the remaining ones.

Another thing you can do with data is exploratory data analysis, using statistics to pull out interesting patterns for further consideration. The workflow for this might look more like:

• Acquire (often multivariate) data from another researcher, source, or experiment.
• Look at its marginal distributions to check your understanding of the system and catch really obvious outliers.
• Maybe use tools like mixture modeling or Box-Cox transformation to clarify marginal distributions.
• Use statistical tools like (linear, logistic, support vector, etc.) regression, PCA, etc., to find patterns in the data.
• Do stuff with the resulting patterns: think up mechanisms, do confirmatory analysis, check literature, show them to other people, etc.

A lot of what you get out of this process will be spurious, but seeing hypotheses that the data seemed to support go down in flames is a good way to convince yourself of the value of confirmatory analysis, and of tools for dealing with this multiple testing problem.

I remember Gelman saying useful stuff like this, but it's been a while since I read that post so I might be mischaracterizing it.

(Ilya, you know all of this, surely at a deeper level than I do. I'm just rhetorically talking to you as a means to dialogue at Capla. Gwern, hopefully my model of you is not too terrible.)