= 27d567bc2d48d9f40e9ccc20ea370b15)
dspeyer comments on Prisoner's dilemma tournament results - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (122)
Can I suggest running a second tournement, including all these bots and new bots that we can examine these bots in writing? After a few iterations, we might beat random. Or we might develop all sorts of signaling quirks, but that would also be interesting.
I like this plan. I'd be willing to run it, unless AlexMennan wants to.
When I first announced the tournament, people came up with many suggestions for modifications to the rules, most of which I did not take for the sake of simplicity. Maybe we should try to figure out what rules will make it easier to write more effective bots, even if that would disqualify many of the bots already submitted to this tournament.
Maybe we should have a prisoner's dilemma meta-tournament in which we define a tournament-goodness metric and then everyone submits a tournament design that is run using the bot participants in the previous tournament, and then use the rankings of those designs.
Wait: does anyone know a good way to design the meta-tournament?
Am I the only one who sees a problem in that we're turning a non-zero-sum game into a winner-take-all tournament? Perhaps instead of awarding a limited resource like bitcoins to the "winner", each player should be awarded an unlimited resource such as karma or funny cat pictures according to their strategy's performance.
No, not the only one. The same complaint was well received in the initial thread.
I'll list some suggestions.
Most of these suggestions are okay, but don't move away from Lisp. A very restricted Scheme (e.g. one way to set variables, one looping construct (I really like foldl, as N shows), etc.) would be good; the lexical scoping and general immutability make it one of the best Lisps for our purposes.
+1 for restricted Scheme. I suggest restricted Python because it is more accessible and would allow more people to participate, but personally I like Scheme a bit more.
On the other hand, many many people were turned off by Scheme last time, and few of them will be reading this post. I'd like to give them a shot, if only to get a more diverse crowd.
On looping constructs: Would that buy us anything? What if people use recursion for looping? Should we restrict recursion?
Question for the audience: would there be a benefit to using a programming language that allows only primitive recursive functions?
I think a programming language that only allows primitive recursion is a bad idea. One common pattern (which I think we want to allow) was for bots to simulate their opponents, which entails the ability to simulate arbitrary valid code. This would not be possible in a language which restricts to primitive recursion.
I think that the bots already submitted should be considered for grandfathering if the rules change to prohibit them.
I also have a specific entry in mind, but lack the expertise to go from 'flowchart' to 'code'.
I think there would more people interested in playing if strategies could be submitted in pseudocode, so that would be great.
For entries that only eval(opponent(x)), for simple x, that seems reasonable. Strategies that rely on deeper analysis would need to look for specific things about the code that I, as a non-programmer, cannot describe accurately.
I would agree that they should compete even if they're not awarded prizes for winning. Because we should at least have a comparison with performance against old bots. More data is pretty much always better.
It should also be trivial to calculate the score both in this heat and against all previously submitted bots.
Bring it on! Just to elaborate, in an iterated 1v1, a defectbot will reliably beat a randombot (4 to 1 if the randombot is unweighted). Or rather, any bot which detects a randombot and then always defects.
However, it's not even necessary to beat the few randombots in a 1v1 to win the tournament, it is sufficient to beat a majority of other bots (including some cooperatebots for easy wins) to get ahead of the pack enough so that the results against the randombots don't matter.