I've played this game, with an actual small prize to give some incentive in favor of cooperating with the experiment. I was surprised at the number of intelligent-seeming people who did not understand that 0 was the "rational" solution. I was unsurprised at the number of people who understood, and submitted answers they knew were irrational just for fun.
This is a bad test of an agreement theorem. There's no reason to believe that participants are motivated to agree, or that their expression of guess is the same as their belief in the "correct" guess.
Proposal for a variation: players may guess any positive real number, and the winner is the one closest to the first quartile of the distribution of answers. This removes both the anchoring effect of the upper bound and the effects of a few jokesters guessing Graham's number and Busybeaver(100) and so on.
It also has the feature of being somewhat more opaque to game-theoretic analysis, at least for me.
I think it's entirely rational to submit a non-zero answer.
I would prefer to win outright, rather than tie, and I think it's safe to assume this is true of more people than just me.
If everyone does the "rational" thing of guessing 0, it will be a big tie.
If anyone guesses above 0, anyone guessing 0 will be beaten by someone with a guess between 0 and the average.
Therefore, a small non-zero guess would seem superior to a guess of zero, to those who value outright wins above ties (EDIT: and don't value a tie as being much better than a loss).
Perhaps I'll write a program to simulate what the best guess would be if everyone reasons as above and writes a program to simulate it...
I haven't decided what my guess will be yet, but if it was instead the average of 95% or so and throwing out the outliers, I'd say "0" without hesitation.
I trust the vast majority to be able to get the right answer on this (and that they trust the vast majority to be able to do the same), but the possibility for a few kooks to screw it up would probably have me submitting a nonzero guess (especially considering that I'm unlikely to be alone in this thinking).
I think there may be some signaling issues to be considered here. We all presumably self-identify to some degree as rationalists, and want to validate Less Wrong as a rationalist community. The more rational a community is, the closer to zero the average guess ought to come. So by guessing zero, lowering the average, you contribute to a signal that Less Wrong is a rational sort of place, and validate your own participation. This could be avoided by not revealing the average guess, but then we've gone away from interesting sociological experiment and into forum games.
This game (along with the prisoner's dilemma and tragedy of the commons) nicely shows how the best choice to make is heavily influenced with how much you know about the other players (and therefore what they vote). If you know that the other players are "rationalists", then you can safely submit 0 (assuming that this hypothetical rational intelligence indeed submits 0). In real world tests you can pretty safely assume that the players are not-perfectly-rational humans. It may also be possible (as you can here) to influence other players.
Interesting. In trying to figure out my guess, I discovered that I care more about losing if everyone else wins, than I care about losing if most other people are going to lose as well. This gives me a greater incentive to pick 0, but in a way that doesn't necessarily reflect my beliefs about the rationality/common knowledge of rationality of this group.
I'd like to play a game with you. Send me, privately, a real number between 0 and 100, inclusive. (No funny business. If you say "my age", I'm going to throw it out.) The winner of this game is the person who, after a week, guesses the number closest to 2/3 of the average guess. I will reveal the average guess, and will confirm the winner's claims to have won, but I will reveal no specific guesses.
Suppose that you're a rational person. You also know that everyone else who plays this game is rational, you know that they know that, you know that they know that, and so on. Therefore, you conclude that the best guess is P. Since P is the rational guess to make, everyone will guess P, and so the best guess to make is P*2/3. This gives an equation that we can solve to get P = 0.
I propose that this game be used as a sort of test to see how well Aumann's agreement theorem applies to a group of people. The key assumption the theorem makes--which, as taw points out, is often overlooked--is that the group members are all rational and honest and also have common knowledge of this. This same assumption implies that the average guess will be 0. The farther from the truth this assumption is, the farther the average guess is going to be from 0, and the farther Aumann's agreement theorem is from applying to the group.
Update (June 20): The game is finished; sorry for the delay in getting the results. The average guess was about 13.235418197890148 (a number which probably contains as much entropy as its length), meaning that the winning guess is the one closest to 8.823612131926765. This number appears to be significantly below the number typical for groups of ordinary people, but not dramatically so. 63% of guesses were too low, indicating that people were overall slightly optimistic about the outcome (if you interpret lower as better). Anyway, I will notify the winner ahora mismo.