It would be very bad if an opponent AI went FOOM. Or even one which optimized for certain types of "fun", say, rescue scenarios.

But consider a game AI which optimized for features found in some games today (generalized):

• The challenges of many games require you to learn to think faster as the game progresses.
• They often require you to know more (and learn to transfer that knowledge, part of what I would call "thinking better").
• Through roleplaying and story, some games lead you to act the part of a person more like who you wish you were.
• Many social games encourage you to rapidly develop skills in cooperation and teamwork, to exchange trust and empathy in and out of the game. They want you to catch up to the players who already have an advantage: those who had grown up farther together.

There are more conditions to CEV as usually stated, and they are hard to correlate with goals that any existing game designers consciously implement. They might have to be a hard pitch, "social innovations" for a "revolutionary game".

If it was done consciously, it's conceivable that AI researchers could use game funding to implement Friendly AGI.

(Has there been a post or discussion yet on designing a Game AI that implements CEV? If so, I must read it. If not, I will write it.)