satt comments on How confident are you in the Atomic Theory of Matter? - Less Wrong

0 Post author: DataPacRat 19 January 2013 08:39PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (80)

Comments (80)

You are viewing a single comment's thread. Show more comments above.

Comment author: gwern 29 June 2013 05:35:13PM 1 point [-]

But (as you've found) I doubt they'd tell you what you want to know, anyway. The problems aren't special epidemiological phenomena but generic problems of causal inference. They just bite harder in epidemiology because (1) background theory isn't as good at pinpointing relevant causal factors

I've found that there's always a lot of field-specific tricks; it's one of those things I really was hoping to find.

This isn't usually that satisfactory, since it's a lot of work and the result often depends on arsepulls.

Yeah, that's not worth bothering with.

(2) controlled experiments are harder to do in epidemiology.

The really frustrating thing about the lithium-in-drinking-water correlation is that it would be very easy to do a controlled experiment. Dump some lithium into some randomly chosen county's water treatment plants to bring it up to the high end of 'safe' natural variation, come back a year later and ask the government for suicide & crime rates, see if they fell; repeat n times; and you're done.

They also can't do things like magically turn a population-level correlation into an individual-level correlation, so I'd guess you're SOL there.

I'm interested for generic utilitarian reasons, so I'd be fine with a population-level correlation.

Comment author: satt 29 June 2013 08:04:58PM 0 points [-]

They just bite harder in epidemiology because (1) background theory isn't as good at pinpointing relevant causal factors

I've found that there's always a lot of field-specific tricks; it's one of those things I really was hoping to find.

Hmm. Based on the epidemiology papers I've skimmed through over the years, there don't seem to be any killer tricks. The usual procedure for non-experimental papers seems to be to pick a few variables out of thin air that sound like they might be confounders, measure them, and then toss them into a regression alongside the variables one actually cares about. (Sometimes matching is used instead of regression but the idea is similar.)

Still, it's quite possible I'm only drawing a blank because I'm not an epidemiologist and I haven't picked up enough tacit knowledge of useful analysis tricks. Flicking through papers doesn't actually make me an expert.

The really frustrating thing about the lithium-in-drinking-water correlation is that it would be very easy to do a controlled experiment.

True. Even though doing experiments is harder in general in epidemiology, that's a poor excuse for not doing the easy experiments.

I'm interested for generic utilitarian reasons, so I'd be fine with a population-level correlation.

Ah, I see. I misunderstood your earlier comment as being a complaint about population-level correlations.

I'm not sure which variables you're looking for (population-level) correlations among, but my usual procedure for finding correlations is mashing keywords into Google Scholar until I find papers with estimates of the correlations I want. (For this comment, I searched for "smoking IQ conscientiousness correlation" without the quotes, to give an example.) Then I just reuse those numbers for whatever analysis I'd like to do.

This is risky because two variables can correlate differently in different populations. To reduce that risk I try to use the estimate from the population most similar to the population I have in mind, or I try estimating the correlation myself in a public use dataset that happens to include both variables and the population I want.

Comment author: gwern 29 June 2013 09:01:16PM 0 points [-]

(For this comment, I searched for "smoking IQ conscientiousness correlation" without the quotes, to give an example.) Then I just reuse those numbers for whatever analysis I'd like to do. This is risky because two variables can correlate differently in different populations. To reduce that risk I try to use the estimate from the population most similar to the population I have in mind, or I try estimating the correlation myself in a public use dataset that happens to include both variables and the population I want.

You never try to meta-analyze them with perhaps a state or country moderator?

Comment author: satt 04 July 2013 08:43:11PM *  0 points [-]

You never try to meta-analyze them with perhaps a state or country moderator?

I misunderstood you again; for some reason I got it into my head that you were asking about getting a point estimate of a secondary correlation that enters (as a nuisance parameter) into a meta-analysis of some primary quantity.

Yeah, if I were interested in a population-level correlation in its own right I might of course try meta-analyzing it with moderators like state or country.

Powered by Reddit Powered by Reddit