2014 iterated prisoner's dilemma tournament results

61 tetronian2 30 September 2014 09:23PM

Followup to: Announcing the 2014 program equilibrium iterated PD tournament

In August, I announced an iterated prisoner's dilemma tournament in which bots can simulate each other before making a move. Eleven bots were submitted to the tournament. Today, I am pleased to announce the final standings and release the source code and full results.

All of the source code submitted by the competitors and the full results for each match are available here. See here for the full set of rules and tournament code.

Before we get to the final results, here's a quick rundown of the bots that competed:

AnderBot

AnderBot follows a simple tit-for-tat-like algorithm that eschews simulation:

  • On the first turn, Cooperate.
  • For the next 10 turns, play tit-for-tat.
  • For the rest of the game, Defect with 10% probability or Defect if the opposing bot has defected more times than AnderBot.

continue reading »
Comment author: lackofcheese 24 September 2014 03:23:53PM 4 points [-]

Also worth checking, out of interest: How does each of the bots perform when playing against itself?

Comment author: tetronian2 24 September 2014 03:54:15PM 11 points [-]

Good question--after running some matches:

AnderBot Cooperates until one of the copies randomly triggers a Defect, then both copies defect for the rest of the match.

CheeseBot, DMRB, SHTFT, SimpleTFTSeer, Switch, TatForTit, TwoFacedTit, and VOFB always cooperate with themselves.

Pip always defects against itself.

Comment author: lackofcheese 24 September 2014 01:27:47PM *  3 points [-]

Your win frequency graph is wrong; it should say 963 for VOFB, not 968.

Anyways, I did some more detailed statistics with the power of
ls | xargs perl -sn0e 'while (/next round:.(\N*)/sg) {print "$1->"}; print "$ARGV\n"' | sort -V
and here's the results I came up with:

963 matches went like this:
ALL->[Cheese,DMRB,SimHatingTFT,Switch,TwoFacedTit,VOFB]->[Cheese,DMRB,VOFB]

32 matches went like this:
ALL->[Cheese,DMRB,SimHatingTFT,Switch,TatForTit,TwoFacedTit]->[Cheese,DMRB,TatForTit]->[DMRB,TatForTit]->[TatForTit]
(13, 19, 30, 31, 33, 40, 42, 54, 74, 119, 137, 203, 236, 272, 301, 303,
321, 326, 343, 400, 438, 476, 516, 526, 539, 608, 626, 815, 823, 827, 941, 977)

3 matches went like this:
ALL->[Cheese,DMRB,SimHatingTFT,Switch,TatForTit,TwoFacedTit,VOFB]->[Cheese,DMRB,SimHatingTFT,TwoFacedTit]->[Cheese,DMRB]
(64, 553, 979)

Match #11 went like this:
ALL->[Cheese,DMRB,SimHatingTFT,Switch,TatForTit,VOFB]->[Cheese,DMRB,SimHatingTFT]->[Cheese,DMRB]

Match #12 went like this:
ALL->[Cheese,DMRB,SimHatingTFT,Switch,TwoFacedTit,VOFB]->[Cheese,DMRB,SimHatingTFT]->[Cheese,DMRB]

Comment author: tetronian2 24 September 2014 02:36:34PM 3 points [-]

Thanks; fixed. And thanks for this analysis! I've added it to the OP.

Comment author: lackofcheese 24 September 2014 10:33:31AM 5 points [-]

Thank you very much for running this and your effort in writing up the results; I'm very happy with how CheeseBot did!

Now I just need to work out how exactly TatForTit managed to exploit me...

Comment author: tetronian2 24 September 2014 02:26:52PM 4 points [-]

Congratulations! If you do work it out, please do post it here--I'm still not 100% sure how TatForTit managed to exploit CheeseBot and DMRB.

Comment author: JoshuaFox 24 September 2014 01:02:16PM 2 points [-]

"Simulate my opponent and figure out if it will defect or if defections will be punished; if so, Cooperate, and otherwise, defect."

Isn't that backwards?

Comment author: tetronian2 24 September 2014 02:25:08PM 3 points [-]

Thanks; fixed.

Comment author: Coscott 23 September 2014 06:23:40AM 1 point [-]

I noticed my bot has a typo. (I do not expect you to change it or anything, just letting you know for your write up if you are confused by my code.)

The two times it says "(fst y,Cooperate)" should have said "(Cooperate, fst y)" I wanted to simulate them next round assuming my previous prediction of their move was correct and I cooperate, not assuming they cooperate and I repeat my last move.

Should have taken you up on the offer to code it for me :( oh well.

Comment author: tetronian2 24 September 2014 02:51:12AM 0 points [-]

Thank you, I was a bit confused, and that did help me understand what you were going for. I'm actually very curious to see how the correct version performs; all of the bots will be open-sourced soon, so you're welcome to experiment. I will probably run some tests as well.

Comment author: Coscott 18 September 2014 01:12:51AM 1 point [-]

How many entries were there?

Comment author: tetronian2 19 September 2014 11:17:19PM 1 point [-]

There are 11 entries; I am very impressed by the amount of effort everyone put into their bots.

Comment author: Coscott 17 September 2014 01:24:20AM 3 points [-]

I am so excited. When will you post results?

Comment author: tetronian2 17 September 2014 08:11:05PM 3 points [-]

I'm glad! It depends on how long it takes me to write up all of the results, but probably within 1 week.

Comment author: tetronian2 02 September 2014 02:23:00AM 3 points [-]

Has anyone else seen the television show Brain Games? It is essentially intro-to-cognitive-biases aimed at the level of the average TV watcher; I was pleasantly surprised by how well it explains some basic biases with simple examples (though I have only seen an assortment of episodes from the 3rd and 4th season). However, most of the material given is not very actionable and is designed more for entertainment rather than self-improvement. Nevertheless, those interested in raising the sanity waterline and/or sparking interest in LW subjects among more average folk than we are might want to take a look at it.

Comment author: V_V 01 September 2014 02:36:31PM 1 point [-]

What is the deadline?

This post says September 1st, while the linked github repo says September 15, 2014 13:59 UTC.

Comment author: tetronian2 01 September 2014 03:04:22PM 1 point [-]

The correct deadline is Sept. 15th--I extended it by 15 days because of an error in the tournament code that went unfixed for two weeks, which delayed some people from writing their bots. I've updated the post to reflect this, thanks for catching it. Anyone who already submitted a bot is welcome to re-submit if they want to make changes to their bot before the 15th.

View more: Prev | Next