But suppose I give people a pre-test, have them play for some minimum time, and then fill out the post-test when they're done. I don't see much in the way for random chance to confound things here: either they know the things needed for solving the tasks, or they don't. If they didn't know enough to solve the problems on the first try, they're not going to suddenly acquire that knowledge in between.

If I get a problem I can't solve I can Google afterwards and read about how to solve the problem. Even if you lock me in a dark room, there the possibility that I recover forgotten knowledge if you give my brain a few hours.

The pretest itself also provides practice. You need a control group, but it would be possible to give the control group nothing to do.