Follow-up to: Fun and Games with Cognitive Biases 

Related to: Rationality Games & Apps Brainstorming, Rationality and Video Games

Answering: Designing serious games - a request for help

Games on Lesswrong

Rational games or game ideas have been discussed or mentioned on lesswrong a few times:

Designing serious games - a request for help proposes to apply games for the rational cause. Fun and Games with Cognitive Biases plays (on a meta level) with concepts. Rationality Games & Apps Brainstorming urges for such games. Rationality and Video Games provides an example. Taboo is explicitly mentioned in http://lesswrong.com/lw/nu/taboo_your_words/. Doing this with scientific concepts could make for a nice casual game at a LW meetup. http://lesswrong.com/lw/3h/why_our_kind_cant_cooperate/ generallly urges for community building mechanism – and (social) games address this. A game that has been discussed extensively on lesswrong is Diplomacy 

 

Introduction

I like to play games that are challenging and intellectually demanding. But I don't like to learn games that will gain nothing except the fun of learning and playing the game.

Games for rationalists should not only be fun. They should also train or teach something beyond the game. Games that don't are parasitic memes.

This post describe areas of rational thought addressed by games. By rationality I mean the LW view.

I list games suitable for getting acquainted with rational thinking. I list areas underrepresented by games and I give an elaborate example of a game suitable to such an area. I propose trying to invent games for these areas and sketch possible strategies.

To repeat I don't mean games that are just played frequently by rationalists or usually considered entertaining by rationalists (though they may). I mean games to train rationalists. Serious Games. To make it easy and fun to become a rationalist (or at least support such a cause).

Games supporting the acquisition of literacy and numeracy are out of scope here even though these are preconditions for more demanding concepts (and games). Also mostly out of scope are social and activity games (though these are be relevant to efficient group building) and games with a focus on motor ability (though such may make spatial or dynamical concepts more clear).

 

Outline

1. I motivate the use of specifically designed games for (rational) education.
2. I illustrate educational game theory with concrete examples.
3. I argue for the applicability of games to some areas of rationality.
4. I present a real game I invented and tested to address the overconfidence bias in particular.

Using and Abusing Curiosity

To use natural curiosity and to direct it to 'worthy' objectives has got the newfangled name edutainment. Well known examples from TV are Sesame Street and Mammutland (which is based on The Way Things Work. Examples of educational games are given below.

The opposite, where natural curiosity is diverted into cognitive dead ends and where attention is collected in trademarked walled gardens is called Marketing. An example is Pokemon where the cute animals engage the curiosity of children which learn lots of names and attributes – but none of these have any lasting cognitive use and all the accumulated knowledge can primarily be used for status and thus directing attention to the trademark owner in the end.

Rational parents have to counter marketing when educating their children. Competing with media is hard because it is omnipresent and not in the best interest of education. Possibly we should lobby against maleducation but that will not help now. Instead we have to turn to games competetive with Pokemon.

Educational Game Theory 

Educational Game Theory names three approaches to educational games:

• creation from scratch by educators

• integrating off-the-shelf games

• creating from scratch by the players (e.g. children)

An example of the first is Ökolopoly (English Ecopolicy) which has a strong focus on grand picture of complex systems. I'd really recommend it for children grade 6 and up (but use the card board version as the simple 'game mechanics' (connected lookup-tables) are inspectable there).

I will provide an example which I invented and tested myself later.

Examples of the second kind are e.g. playing Taboo with math words or playing Liar's Dice and calculating expected values. Much more will be listed below with a focus on specific concepts.

That children invent games all the time is no surprise. Because one natural aspect of games is that they support or stimulate learning most games invented by children are educational by nature.

To illustrate this I'd like to describe a game my oldest son (9 years) invented:

He calls it 3D-computer and it consists of a 'user interface' made of paper and plastics, a number of 'input devices' and a large playing area made of taped together sheets of paper with a plan of a city. The player (usually one of his younger brothers) sits at the 'controls' and e.g. steers a car on a racing track thru the city. He has to press left and right (and say so loud) and my oldest son will act out the operation of the 3D-computer by moving the cars accordingly.

Obviously this is modeled after real computer games but it requires significantly more abstraction and cooperation of the players. To build the 'game' he had to plan it and build corresponding pieces. He created a 'user interface' consisting of menues and sub menues (drawn on paper) for the possible games and options that can be played with the 3D-computer. It makes the logic of the game – its concepts – clear both in the planning and realization and in the acting too.

 

Concepts in Games

Concepts in Games

Any normal game requires some rational thinking, but there are some areas of rational thought that are less covered by games than others.

Game Theory

This lends itself naturally to games with clear concepts. The simplified games analysed in game theory can be easily readapted to playable games or parts thereof. Some of these are directly playable for school children. I played lots of these in a math course.

Game theoy obviously applies to most games, but the basic min-max principle is very clearly present in games that have a measurable of advantage; I identify the following kinds

 

1. Distance of pieces from a finish on a game board (e.g. in the simple childrens games above
2. Some in-game currency (examples: Monopoly)
3. A winning criteria that includes a tally (example: Siedler von Catan)

In these cases there may be strategic effects that outweigh the measurable but in the long run 'more is better' and a higher value predicts a win well.

 

Probability Theory

Classical games with dice or shuffled cards surely build some intuition for probability theory which is present in most games in so far as some most games need some controlled random variables (aka dice or shuffled cards) to support the game semantics.

It is also present when simulating (or testing) games; the game state changes due to the game rules are implied stochastic processes.
Probability theory is relevant as games with numerous regular random events satisfy criteria of theorems of large number.

Examples for dice games with simple rules and clear concepts are Cross And Circle, Snake and Ladders. These illustrate basic concepts and reflection about strategies using solid terminology can teach children a lot about probability.

Intuition about different distributions resulting from superposition and their likelihood is fairly clearly present in Yahtzee. 
Intuition about the law of large numbers (and updating due to new information) can be found in an entertaining way in Liar's dice.

Decision theory

Games that are not based on skill necessarily involve decisions by the players. These fall into three categories

• Decisions under certainty – games are seldom certain, but in some cases decisions can be modeled as if the game environment were fixed and then maximizing over multidimensional ratings of the game state (some measurables like win points)

• Decisions under risk – when some decisions involve probabilistic losses or gains

• Decisions under uncertainty – if some decisions have uncertain consequences either due to unknown probabilities of game events or due to the other players.

Clear decision strategies or concepts are not so easy come by. Most games imply lots of decisions but effective strategies are seldom obvious.

Games where subjective valuations are rampant are trading cards (e.g. Pokemon or currently Star Wars) these are no simple trades but often involve a risk (gaining certain cards vs. losing some other cards due to a random or skill based process). What is unclear is whether there are any lessons learned from doing these games as the decisions are mostly individual and the conditions of the transactions vary from trade to trade so general insights are unlikely.

Cognitive Science

Cognitive biases can be harvested for game aspects worth to address. I will demonstrate this with the Overconfidence Effect and with miscalibration of probabilities particular which is the main focus of the 'estimation game' presented below. I will cover other biases in more detail in a separate post.

Humans are very good at detecting cheaters this is well known since the work of Cosmides and Tooby.This applies to real life as well as games. It would be interesting to develop games that try to move smoothly from obvious rules to abstract rules and try to train to perceive this as cheating.

Math and Logic

Math in general plays a role in most games. Correct logical inferences (even if done by intuition) train propositional logic (though a game which involves using modus ponens on unusual conditions would surely be a good idea).

A very nice and pure example using Set Theory even in its name is Set. 

Looneylabs has a lot of games (or game material) suitable for logic. A very nice game building on this is Zendo.

 

Estimation Game 'Who guesses best'

Now I'd like to present an example of a game which was created to explicitly addresses a cognitive bias – the overconfidence bias – which is hard to overcome even for scientifically schooled persons explicitly briefed in this bias.

When I read chapter 21 by M. Alpertand H. Raiffa in Judgement under uncertainty - heuristics and biases 1982 by Kahneman et al (pages 294 – 305) see this Google Books link and this less wrong review.

I couldn't believe that it could be that hard to calibrate well and I decided to test this on myself and on others in a comparable setting (kind of privately reproducing the study). This developed into a game that was since played about 5 times with a total of about 20 person and 30 questions (thus much less than in the cited study). The results were as expected – basically. A significant trend was visible in the test games and also in the main game rounds during a large birthday party. I had improved the 'game chart' and scoring rules and this made the effect clearer and the calibration of most players improved quickly. The game is simple enough to be played by a smart eight year old.

The game chart can be downloaded here (on page 2 in english).

Estimation Game Rules

This is basically an estimation game – but with some extensions.

The Questions

First a number of questions about quantities to guess are needed.

Each quantity should have an exact value that has a believable source but is somewhat unusual such that most players will not know the value.

In a scholarly context the question should name a study or method and date of determination of the quantity. E.g. for the question “How many residents live in Hamburg, Germany?” the context could be: “from wikipedia: official census of 2012 determined via population register”. And for the trivial question “How many lentils are in this jar?” the context could be: “This morning I filled the jar with green lentils, weight the content, counted and weighed a sample and calculated the total number.”

Example questions:

• Number of steps to the next underground station.

• Number of lentils in a jar.

• Amount of rain per square meter per year in Jerusalem in 2001.

• Number of files on my PC as reported by ls -R this morning.

• How long (days) was the first voyage of James cook into the south seas?

• Total egg production in th U.S. In 1965. (this is from the Alpert Study)

• Toll collections of the Panama Canal in 1967. (dito)

The questions can be provided by the host (which has the disadvantage that host cannot take part in the game for most questions). The better idea is to have each player provide one to three questions.

Guesstimation

Each question is now dealt with as follows:

• The question is read out loud. If needed, detail questions about the context of the question are answered.

• All players individually guesstimate the true value of the quantity in question. The number is written into the field in the middle below the green area. If the number is from one of the players he/she writes the correct number as his 'guess'.

• Each player now considers what the other players might estimate. Values deemed to be definitely out of the range become the min/max values and written below the red/yellow border. Values deemed to be typical form the majority range and are written below the yellow/green border.

• Points are awarded as follows:

  • For the best guess you get N points, for the next-best N/2, then N/4 (rounding down).... If multiple players are equally near, points are added and split. Numbers supplied by a players are excluded from scoring.

  • For the green area you get N/2 points if it contains exactly N/2 numbers (including your own) for each more or less you get one point less (rounding up).

  • For the yellow area the player with the smallest range which still contains all guesses gets N+1 points, the next-best N/2+1 (and so on, rounding down).

  Numbers falling on a boundary are scored to the players advantage.

Example:

• For the number of lentils in the jar you estimate 1300. And you guess that half of the guesses fall within 500 and 2000. You also assume that nobody will believe less then 100 or more than 10000.

• It turns out that the other 6 players guessed 750, 1000, 1500, 2000, 2820, 4000.

• You write 750, 1000, 1500 and 2000 into your green area. You write 2800 and 4000 into the right yellow area. Congratulations: No numbers fell into a red area.

• The correct answer is revealed: 2800. You can circle it in the green area.

• The player guessing 2000 (distance 820) gets 7 points, the player guessing 4000 (distance 1180) gets 3 points floor(7/2), the player guessing 1500 (distance 1320) gets 1 point floor(7/4). For your 1300 you and the remaining players get nothing (7/8<1).

• You have got 4 numbers in your green area (you decide to include 2000). 4=ceil(7/2) is the optimum number of entries so you get 4 points. Had you excluded 2000 you'd got 3 points. If only your own number or if all 7 numbers were included you'd got 1 booby point.

• All the numbers are in your min-max-range, so you get at least 1 point. Lets assume that your range of 9900 is the second smallest of those containing all numbers. This nets you an additional floor(7/2)=3 points.

 

May sound complicated. A summary of these rules is part of the game chart. The chart is mostly self-explanatory and even school children can fill-out the chart after one round of explanation.

Gaming the Rules

It is possible to game the rules by e.g. using absurd guesses to bomb the max-range of the other players. The scoring is chosen to make this a losing strategy. Good guesses and good majority bounds score higher than the points reaped from being the only one with a valid max-range. And players can hedge against bombing nonetheless (in a way these are black swan events and thus interesting in their own right).

Not quite Overconfidence

This game doesn't directly address the Overconfidence Bias. To do so the players would have to guess the range of their own guesses. Getting the range of ones own guesses via this kind of game takes much longer (times the number of players). The game works so well because it forces you to take the outside view. You have to consider what the other players might not know and then getting immediate feedback about ones own performance via

a) Distance from the truth of your own guess.

b) In extreme cases exclamation from the other players whose ranges were shredded.

It is critical to make the leap that your own guesses are as (over)confident as those of the others.

Variants

It is quite possible to play a stricter version where you guess your own ranges with the same game chart. The following differences are necessary:

1. Instead of the majority range you have to guess the range into which the true value will fall mostly. Mostly meaning on half of all questions this range should actually contain the value.

2. Instead of the maximum range you have to guess the range into which the true value will always fall.

1. Don't write the numbers of the other players into the colored area.

2. Use a singe color strip for all of your own guesses. Just make a mark in the area where the actual value lay in the end.

3. Points are awarded only at the end of the game.

This has the disadvantages of taking much longer and with the above simple rules it is too easy to game the rules (e.g. by using absurdly large or small ranges at the end of the game to ensure the required counts).

But this can be used after a few games to test whether you can make the leap to take the outside view on your own guesses. It can be done alone. Just think of a few arbitrary quantities and then research them later (on danger of choosing only quantities you can provide sensible estimates for).

Anchoring

Obviously your ranges are anchored to your guess. It is possible to add a small tweak to change the anchoring (disclaimer: I didn't try this): Allow the person knowing the correct answer to supply any example number and say it out load beforehand. This will provide another anchor (obviously anchored to the correct number somehow).

Motivation

The test-games were fun. OK. The game may not be fun for everybody. My acquaintances are mostly smart and well-educated. But even my 8 year old son liked it (and scored in the middle range of 10 players). It is fun especially if people come with their own individual questions. The resulting discussions about individual (mis)reasoning is also often insightful and a nice ice-breaker.

The performance of the players definitely improved.

Friends who are teachers asked me for the material and used it in highschool.

I hope this is is step into the right direction.

I'd like to close with an anecdote: I still try to win any game that I play and my friends know it. Sometimes, especially after I play unusual or when explaining tricks and then losing anyway I get remarks about it obviously not helping. To that my reply mostly is: I maximize my chances on infinitely many runs. So in the end I learned to lose.