Also, the behaviour at the very end is very much "oh we didn't expect that to happen forever when we wrote that utility function"
It was published on April 1.
But he certainly expected the "increment any counters" objective function would not map to perfect play, at least by the time he got to Tetris. It was more of a case of "well, this algorithm is probably not going to beat any of these games, but I'm not sure of all the ways that it will fail."
"Pretty simple" algorithm playing games quite impressively.
http://www.youtube.com/watch?v=xOCurBYI_gY
First, this is awesome - enjoy!
Paper here http://www.cs.cmu.edu/~tom7/mario/mario.pdf
One interesting observation made by Tom Murphy is that the AI found and exploited playable bugs in the game not (commonly) known to human players. I think it's a good example to have available suggesting what a really smart AI might look for to win.