While I'm at it, here are links to a bunch of other games that require some degree of thinking outside the box and adapting to changing rules:
Factory Balls and Factory Balls 2
Easy. Each level introduces new puzzle pieces, but no dramatic changes in the rules. The solutions are all inside the box, once you figure out what the rules are.
Easy. Requires solving some puzzles without any hints what the puzzle is or what a solution would look like. Solvabe just by trying random things until something happens.
Medium. A series of puzzles, each of which has different rules.
Medium. A series of puzzles, each of which has different rules.
Medium. Each level introduces new puzzle pieces, but no dramatic changes in the rules. The solutions are all inside the box, but you have to figure out how to put them together.
Medium. Each level introduces new puzzle pieces, but no dramatic changes in the rules. The solutions are all inside the box, but you have to figure out how to put them together.
Hard! A series of puzzles, connected by other puzzles, each of which have different rules, and most of which have a counterintuitive solution. Hints about the solutions are cleverly hidden in the game. Hint: Gur cevagfperra ohggba vf lbhe sevraq. (rot13'd)
Hard. A series of puzzles, each of which has a deliberately counterintuitive, and often malevolent, solution. Don't worry if you can't solve all of them, some of the solutions require specific computer hardware or softwae to win.
The Impossible Quiz and The Impossible Quiz 2
Almost Impossible. A series of quiz questions and other challenges that have deliberately counterintuitive solutions. Some of the quiz questions are solvable only by trial and error. Some of the challenges require extremely fast reflexes. Many of the puzzles are blatantly evil. Do not expect to win this. You have been warned.
The rest of these games don't really fit with the theme of thinking outside the box and adapting to changing rules, but are unique enough to include in the list anyway:
Medium. A unique game, but the rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Medium. This game doesn't really belong in this list. The rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Medium. This game doesn't really belong in this list. The rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Shift, Shift 2, Shift 3, and Shift 4
Medium. This game doesn't really belong in this list. The rules don't change much, except when new puzzle elements are introduced. The solutions are mostly inside the box.
Medium. Actually, this doesn't belong in this list at all. The rules don't change, and the solutions are all quite literally inside the box.
There are plenty of other good puzzle games I could link to, but they didn't really fit with the theme of thinking outside the box and adapting to changing rules.
Question: Would it be inappropriate to put this list somewhere on the Less Wrong Wiki?
These kinds of games just remind me of this Monty Python skit. There's no rules by which to play, so you're just trying to guess what the author is thinking.
A few years ago I came across The Logic of Failure by Dietrich Doerner (previously mentioned on LW) which discusses cognitive failures in people dealing with "complex situations".
One section (p.1 28) discusses a little simulation game, where participants are told to "steer" the temperature of a refrigerated storeroom with a defective thermostat, the exact equation governing how the thermostat setting affects actual temperature being unknown. Players control a dial with settings numbered 0 through 100, and can read actual temperature off a thermometer display. The only complications in this task are a) that there is a delay between changing the dial and the effects of the new setting; b) the possibility of "overshoot".
I found the section's title chilling as well as fascinating: "Twenty-eight is a good number." Doerner says this statement is typical of what participants faced with this type of situation tend to say. People don't just make ineffective use of the data they are presented with: they make up magical hypotheses, cling to superstitions or even call into question the very basis of the experiment, that there is a systematic link between thermostat setting and temperature.
Reading about it is one thing, and actually playing the game quite another, so I got a group of colleagues together and we gave it a try. We were all involved in one way or another with managing software projects, which are systems way more complex than the simple thermostat system; our interest was to confirm Doerner's hypothesis that humans are generally inept at even simple management tasks. By the reports of all involved it was one of the most effective learning experiences they'd had. Since then, I have had a particular interest in this type of situation, which I have learned is sometimes called "experiential learning".
As I conceive of it, experiential learning consists of setting up a problematic situation, in such a way that the students ("players") should rely on their own wits to explore the situation, invent ways of dealing with it (sometimes by incorporating conceptual tools provided by an instructor), and test their newfound insights against the original problem - or against real-world situations. My preferred setting for experiential learning is a small-group format, with individual or group interaction with the situation, and group discussion for the debrief.
In experiential learning there is no "right" or "wrong" lesson to be taken from a game or simulation. Everything that happens, not just the ostensible game but also the myriad meta-games that accompany it, is fodder for observation and analysis. Neither is realism a requirement for experiential learning; it is an understood convention of the genre that such games present an abstraction of some "real world" situation that necessarily deviates from it in many respects.
The important part of an experiential learning situation is the debrief. In the debrief, you initially refrain from drawing conclusions about the experiment. The first thing you want from the session is data. A good question to ask is "What happened in this session that stood out for you?"
Because you want to map the game back to the real world, perhaps in unforeseen ways, another thing you want from the session is analogies. A good question to ask is "What did the experiences of this session remind you of?"
For learning to take place there should also be some puzzles arising from either the observations made during the game, or their transposition to real life. For instance, your preexisting mental model - derived from real life interactions - would have led you to different predictions about the game.
The intended outcome of experiential learning is for students (and, sometimes, teacher) to construct an updated mental model that resolves these tensions and can be transposed back to real world situations and applied there. A constructivist approach doesn't expect students to draw exactly the same conclusions as the teacher, even when the teacher makes available the ingredients out of which students build their updated model. Knowledge obtained in that way is more truly a part of you - it sticks better than anything the teacher could have merely told you.
An experiential learning game focusing on the basics of Bayesian reasoning might be a valuable design goal for this community - and a game I'd definitely have an interest in playing. Such is my "hidden agenda" in publishing this post...
Any takers ?