Normal_Anomaly comments on A fun estimation test, is it useful? - Less Wrong

5 Post author: mwengler 20 December 2010 09:09PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (49)

You are viewing a single comment's thread. Show more comments above.

Comment author: Normal_Anomaly 23 December 2010 01:54:39AM *  0 points [-]

1) I like having at least some data; I still found myself using all 10 options at least once. That is, the test still relied to a large extent on my prior knowledge.

2) You're right about this. I tried and they don't; guessing 50% every time got me a perfect. I don't know enough about designing these things to make one with a log scoring rule, but it would definitely be nice to see one.

3) Ooh, that is weird. The gridlines don't seem to mean as much as the actual numbered labels; taking them off would make this go away.

It seems like neither of these tests is able to measure both calibration and discrimination.