= 27d567bc2d48d9f40e9ccc20ea370b15)
philh comments on Announcing the 2014 program equilibrium iterated PD tournament - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (61)
You actually cooperate in this case.
Quick analysis: you're going to defeat CooperateBot 500 points), lose against DefectBot (0 points), and tie against TitForTatBot (250 points from alternating D/C and C/D). Against RandomBot, you are also RandomBot, both of you scoring 225 on average.
When you simulate MirrorBot, the infinite recursion makes ver time out, so you cooperate. So MirrorBot cooperates against you as well (300 points). SmarterMirrorBot and JusticeBot both time out as well. SmarterMirrorBot can't work out what you'll do, and cooperates (300 points). JusticeBot may or may not be able to work out what you'll do against CooperateBot, and defects either way (0 points).
But I think TitForTatBot should beat that, at least: 300 against CooperateBot, 99 against DefectBot, 300 against JusticeBot, 223.5 against RandomBot, and all other scores the same.
So, I'm puzzled too, if TrollBot is getting the highest score in the first round.
Whoops. Effect goes away if I fix it, too.
Here are the average results for the first round:
http://pastebin.com/qN5H2A25
For some reason, TrollBot always wins 500 / 0 against SmarterMirrorBot. DefectBot actually beats TrollBot by a narrow margin (1604 - 1575 = 29 points) on average, but there is quite a bit of randomness from RandomBot, so TrollBot often comes out ahead of even DefectBot, and they both come out over 100 points ahead of the next bot (TitForTatBot).
Since I built TrollBot as a sanity check on my actual bot to make sure that it would defect against TrollBot, I was definitely surprised by the fact that TrollBot outperformed not only my attempt at a serious bot, but also most of the other bots... :/
Oh! You're also running your opponent playing a game against MirrorBot, not against TrollBot.
Which I still don't understand... you run SMB versus MB, time limit 10000. SMB runs MB versus MB, time limit 10000. MB versus MB times out, which meas SMB runs for > 10000 us, which means that SMB should time out and you should cooperate.
Meanwhile, SMB runs you versus MB, time limit 10000. You run MB versus MB, time limit 10000. That times out, so you time out, so SMB cooperates.
But you're defecting, which means that you're running SMB versus MB to completion, which seems like it shouldn't happen.
That does seem exploitable, if one can figure out exactly what's happening here.
Yes, it's an issue with how the "time" command is set up. Basically, timeouts are done via threads an exceptions; there's a detailed description of the "timeout" function in System.Timeout here: http://chimera.labs.oreilly.com/books/1230000000929/ch09.html#sec_timeout
However, tetronian2's "time" command catches all exceptions, which means that if multiple timeouts are nested, you can get some pretty strange behaviour.
In the TB vs SMB scenario, here's what happens:
1. The world runs TB vs SMB
2. TB runs SMB vs MB (timeout #1, 10000us)
3. SMB runs MB vs MB (timeout #2, 10000us)
4. Timeout #1 triggers an exception, which is caught by timeout #2 and handled, causing (3) to return Nothing.
5. Before the exception for timeout #2 can actually trigger, (2) finishes normally and kills the timeout thread; this means that (2) return Just Cooperate.
6. Since (2) returned Just Cooperate, TrollBot returns Defect.
I've brought this up as an issue on the GitHub issues page here: https://github.com/pdtournament/pdtournament/issues/3
I've suggested a solution that fixes this problem, but unfortunately race conditions are ultimately still present.
Thank you very much for catching this and for explaining it in detail here and on the github page; I'm going to play around with your proposed solution today and brainstorm any other ways of dealing with this.