We shouldn't duplicate work; I'm currently writing up a tournament simulator myself. I'm confident on variable match length, evolution, noisy responses, cheap talk rounds, and natural selection population growth tournaments. I am less confident on simulating opponents. For submitted strategies, I was prepared to accept code, pseudocode, and English descriptions, and then translate them into coffeescript myself. Probably I will also ask for some example games to test my translations (e.g. "My bot should cooperate 1-98 and defect 99, 100 against pure tit-for-tat").
So your scripting language sounds particularly interesting, as well as your ideas for simulating opponents. I gather the two are tied together. PM me for details!
ETA: I offer to save you the cost of overcoming akrasia and lack of time.
I was prepared to accept code, pseudocode, and English descriptions, and then translate them into coffeescript myself.
I did it last time and it was pain in the arse. It's fun to code one PD agent, not so much when they are forty and you must follow descriptions which you are not sure that you understand correctly.
I have written an interpreter which evaluates functions written in a lisp-like prefix notation, which was originally intended to be used in a program evolving automatic traders using genetic algorithms and real stock exchange data - the traders...
Last year, there was a lot of interest in the IPD tournament with people asking for regular events of this sort and developing new strategies (like Afterparty) within hours after the results were published and also expressing interest in re-running the tournament with new rules that allowed for submitted strategies to evolve or read their opponent's source code. I noticed that many of the submitted strategies performed poorly because of a lack of understanding of the underlying mechanics, so I wrote a comprehensive article on IPD math that sparked some interesting comments.
And then the whole thing was never spoken of again.
So now I'd like to know: How many LWers would commit to competing in another tournament of this kind, and would someone be interested in hosting it?