= 27d567bc2d48d9f40e9ccc20ea370b15)
Houshalter comments on 2016 LessWrong Diaspora Survey Analysis: Part Four (Politics, Calibration & Probability, Futurology, Charity & Effective Altruism) - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d){kind=logo}
Comments (27)
I didn't design the questions and those are the official answers. And it does seems correct to me, that it should include all bills ever printed and not just those currently being printed.
I'm really not sure how to do your second point. I could fit all the answers into a normal distribution sure, but what information does that give me for any specific individual? It doesn't really tell me what their true probability of getting the question correct was, which I can already get from the percent of people that answered each question correctly.
The third idea is interesting, comparing people who got the same number of answers right. But it still does reward luck and prior knowledge. As I showed, people have indistinguishable probabilities of getting each question right, all that differs is how overconfident or underconfident they are.That model seems to produce the best correlations as well.
Agree with Luke about the Hamilton questions. I read it about current ones. If it meant "has appeared on", it should say "has appeared on", not "appears on". While certainly the latter can be read as all the ones he's ever appeared on, the more natural interpretation to me means those currently being printed.
You can probably get some idea to what extent it was interpreted this way by looking at the size of the answers. I'd say, if we assume people have some idea how American currency works, then 0-1 probably indicates a "present" interpretation, 3 or more will almost always indicate an "ever" interpretation, and 2 is hard to tell from. But that is assuming people have some idea of how American currency works.
1) Really?
It's the present tense that throws me. I would expect the question to be 'has appeared'. Whatever.
2) That's not what I meant. I mean, you can turn each individual person's prediction/probability pair into a gaussian curve. It is centered on their answer with width such that a 10 year window contains that much probability. You can then use that to get the probability this distribution - and thus, by proxy, the respondent - assigns to the actual year.
3) On such a small data set you can't get rid of luck, let alone differences in knowledge. I think that by picking out people who got the same number correct you do a pretty good job of de-confounding that. It cuts sideways across the bins in the 'mean % correct' vs 'mean % confidence' graph which showed flat performance across confidence, in a way that you can't do straightforwardly otherwise.