satt comments on How confident are you in the Atomic Theory of Matter? - Less Wrong

0 Post author: DataPacRat 19 January 2013 08:39PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (80)

You are viewing a single comment's thread. Show more comments above.

Comment author: satt 29 June 2013 08:04:58PM 0 points [-]

They just bite harder in epidemiology because (1) background theory isn't as good at pinpointing relevant causal factors

I've found that there's always a lot of field-specific tricks; it's one of those things I really was hoping to find.

Hmm. Based on the epidemiology papers I've skimmed through over the years, there don't seem to be any killer tricks. The usual procedure for non-experimental papers seems to be to pick a few variables out of thin air that sound like they might be confounders, measure them, and then toss them into a regression alongside the variables one actually cares about. (Sometimes matching is used instead of regression but the idea is similar.)

Still, it's quite possible I'm only drawing a blank because I'm not an epidemiologist and I haven't picked up enough tacit knowledge of useful analysis tricks. Flicking through papers doesn't actually make me an expert.

The really frustrating thing about the lithium-in-drinking-water correlation is that it would be very easy to do a controlled experiment.

True. Even though doing experiments is harder in general in epidemiology, that's a poor excuse for not doing the easy experiments.

I'm interested for generic utilitarian reasons, so I'd be fine with a population-level correlation.

Ah, I see. I misunderstood your earlier comment as being a complaint about population-level correlations.

I'm not sure which variables you're looking for (population-level) correlations among, but my usual procedure for finding correlations is mashing keywords into Google Scholar until I find papers with estimates of the correlations I want. (For this comment, I searched for "smoking IQ conscientiousness correlation" without the quotes, to give an example.) Then I just reuse those numbers for whatever analysis I'd like to do.

This is risky because two variables can correlate differently in different populations. To reduce that risk I try to use the estimate from the population most similar to the population I have in mind, or I try estimating the correlation myself in a public use dataset that happens to include both variables and the population I want.

Comment author: gwern 29 June 2013 09:01:16PM 0 points [-]

(For this comment, I searched for "smoking IQ conscientiousness correlation" without the quotes, to give an example.) Then I just reuse those numbers for whatever analysis I'd like to do. This is risky because two variables can correlate differently in different populations. To reduce that risk I try to use the estimate from the population most similar to the population I have in mind, or I try estimating the correlation myself in a public use dataset that happens to include both variables and the population I want.

You never try to meta-analyze them with perhaps a state or country moderator?

Comment author: satt 04 July 2013 08:43:11PM *  0 points [-]

You never try to meta-analyze them with perhaps a state or country moderator?

I misunderstood you again; for some reason I got it into my head that you were asking about getting a point estimate of a secondary correlation that enters (as a nuisance parameter) into a meta-analysis of some primary quantity.

Yeah, if I were interested in a population-level correlation in its own right I might of course try meta-analyzing it with moderators like state or country.