Setting and story
So after a bit of thought, I was thinking that I could resurrect the setting for a fantasy novel that I started writing once, but never got very far in: The City of Light and Fire. The second part of that story opened as follows:
The basin had been a mold, shaping me to look roughly like a human. But I was still far from perfect.
The figures that carried me put me down on a long stone bench, and then left. There were other shapes on both sides of me, other early-stage embryos. I did not yet understand anything.
Time passed, and the lava I was made of grew more solid. My innermost parts were still hot and liquid, but I had a firm outer crust. When I had become hard enough to be worked on, the mason had me brought to him. He studied me for a long time, examining me from every direction and seeking out any imperfections. Whenever he found one, he reached for his hammer. Gradually, he shaped me into a man.
After the mason was done with me, I was taken to the clay maker. The beings carrying me were cautious, for the mason had opened holes from which my innards might spill. I don't know whether they spilled any, but when I reached the clay maker, I was still viable.
He studied my shape, and then molded a layer of lifeclay around me. It was much softer than lava was, and more sensitive to heat. The clay maker filled the pair of holes the mason had made, fashioning there eyes. Below them he made a mouth, and on their sides a pair of ears. The clay was as good in shaping heat as the walls of the towers were, if not better. It collected warmth and funnelled it deep into my core. My eyes had been made with particular care, and it now that I slowly began to see.
From the clay maker, I was carried to the edge of a great hall. They placed me next to the other infants, on a belt of heatstone close to the wall. The stone burned hot, keeping us sated. I rested there, together with the others. We watched and listened to the things happening in the hall, enjoyed the ever-shifting flows of warmth inside the stone. For a long time, we remained still.
As in the original story, the main character in the game would be a creature who had just recently been brought into existence for an unknown purpose. He (she?) comes to existence with some basic skills, such as knowing how to talk and walk, but knows little besides that. His creators clearly have a purpose in mind for him, but don't care to tell him very much about it. Other creatures that have been created in a similar fashion might have clues of just what exactly is going on, as do others who he meets on missions that he is sent on... but he has to figure out what exactly it is that can be reliably inferred from the various claims that the different creatures make.
Advantages: This would instantly set up a mystery (just what is this place? why was he created? what's going on?) to make the player curious, and gathering various clues about his origin, as well as completing different missions that the character was sent on, could serve as short-term goals for the player to pursue. An added benefit of making the main character non-human is that the whole belief network could be diegetic - perhaps connecting the different pieces of evidence would concretely build up a physical belief network inside his body, and by closing his eyes he could somehow see and manipulate that. Such a setting would also allow us to question the beliefs and attitudes that were commonly believed and taken for granted within the setting, without getting into the politics-is-the-mindkiller territory that having a game about questioning commonly-accepted real life beliefs would involve.
Here's an old description of the setting:
I earlier described that as "a New Weird-ish story about a corporation whose different departments are in semi-open war with each other. The main character is a gargoyle created by Product Development to infiltrate Marketing and kidnap some of their harpies, to be pressed into slave labor to boost Product Development's morale." Though actually sirens are a closer match to what I had in mind than harpies - I mixed up the two.
Anyway, it's this weird dark fantasy setting with a huge, black stone monolith in the middle of a city. The Corporation exists within the various halls and caverns of the monolith, and its workers produce and sell various goods to the people in the city. There are also other cities and in them other corporations, which have a bit of an influence in this city as well, but mostly they remain in the background.
I don't actually have very many details worked out yet, mostly just concepts and images. Here are some of them:
In the city around the monolith, there are sentient stone towers. They are filled with hot lava, and they communicate with each other by selectively making their outer shells more or less transparent, shining light around them. They can choose to voluntarily let out a small amount of their lava and let it cool, creating a new being separate from themselves that serves them. Or somebody might break their shell and take some lava by force to create a new being, which is how the story's main character was created. The towers have their own interests, and they often do not look kindly upon the Corporation.
The ultimate leaders of the Corporation are the Owners, who never intervene directly. The Owners are what seem to be clouds of ever-burning gas above the monolith, who communicate their desires through patterns in the fire.
On the roof of the monolith are the Prophets, monsterously shaped philosophers who sit on top of large stone pillars and watch the sky, interpreting the messages of the Owners. They keep making a constant humming sound, which communicates their interpretation of what the Owners want. This is in turn interpreted by the Scribes, who sit at the feet of the pillars and write down what they hear. A constant stream of messengers takes what the Scribes have written down and relays it to the various departments.
When the department heads receive their orders from the Scribes, they too apply their own interpretation, seeking to follow the instructions in the way that best benefits them personally. The original orders being distorted after going through several steps in the chain, the department heads are free to do almost anything they want. It is because of this personal benefit-seeking that the various departments of the Corporation are in a constant conflict with each other.
Information only seems to pass down from the Owners to the Corporation. If information does pass up to the Owners, nobody knows how.
Even the department heads do need to follow some rules, however. In the middle of the monolith there is the Timetable, a wall which shows the past and a possible future. In particular, it shows various things which the Corporation might be able to achieve in the future. If those things are indeed achieved, all is well.
If they are not, cracks begin to appear in the Timetable. This is a bad thing, for the Timetable has been built to seal a rupture to another reality, in which terrible beings live. If the Timetable ever breaks, the beings will burst out and destroy the monolith and everyone who lives in it. Because of this, the department heads cannot let their infighting get so serious that the Timetable won't hold. It is said that the Timetable, and the rupture behind it, were created by the Owners for this very purpose.
The titular Fundamental Question could then also refer to the task of figuring out just what it is that the Owners really want (which could also be the real reason that the character was created).
A disadvantage of this setting pitch is that there might be a risk of the setting becoming too weird and different to effectively relate to, or for people to very naturally transfer the lessons of that setting into real life. But that could avoided by focusing more on the surrounding city, which could be more normal, or something. Thoughts?
Hmm, on other thought a more realistic modern setting might after all feel like a better fit for this kind of game. Hmmh.
I was much impressed when they finally came out with a PC version of DragonBox, and I got around to testing it on some children I knew. Two kids, one of them four and the other eight years old, ended up blazing through several levels of solving first-degree equations while having a lot of fun doing so, even though they didn't know what it was that they were doing. That made me think that there has to be some way of making a computer game that would similarly teach rationality skills at the 5-second level. Some game where you would actually be forced to learn useful skills if you wanted to make progress.
After playing around with some ideas, I hit upon the notion of making a game centered around the Fundamental Question. I'm not sure whether this can be made to work, but it seems to have promise. The basic idea: you are required to figure out the solution to various mysteries by collecting various kinds of evidence. Some of the sources of evidence will be more reliable than others. In order to hit upon the correct solution, you need to consider where each piece of evidence came from, and whether you can rely on it.
Gameplay example
Now, let's go into a little more detail. Let's suppose that the game has a character called Bob. Bob tells you that tomorrow, eight o'clock, there will be an assassination attempt on Market Square. The fact that Bob has told you this is evidence for the claim being true, so the game automatically records the fact that you have such a piece of evidence, and that it came from Bob.
(Click on the pictures in case you don't see them properly.)
But how does Bob know that? You ask, and it turns out that Alice told him. So next, you go and ask Alice. Alice is confused and says that she never said anything about any assassination attempt: she just said that something big is going to be happen at the Market Square at that time, she heard it from the Mayor. The game records two new pieces of evidence: Alice's claim of something big happening at the Market Square tomorrow (which she heard from the Mayor), and her story of what she actually told Bob. Guess that Bob isn't a very reliable source of evidence: he has a tendency to come up with fancy invented details.
Or is he? After all, your sole knowledge about Bob being unreliable is that Alice claims she never said what Bob says she said. But maybe Alice has a grudge against Bob, and is intentionally out to make everyone disbelieve him. Maybe it's Alice who's unreliable. The evidence that you have is compatible with both hypotheses. At this point, you don't have enough information to decide between them, but the game lets you experiment with setting either of them as "true" and seeing the implications of this on your belief network. Or maybe they're both true - Bob is generally unreliable, and Alice is out to discredit him. That's another possibility that you might want to consider. In any case, the claim that there will be an assassination tomorrow isn't looking very likely at the moment.
Actually, having the possibility for somebody lying should probably be a pretty late-game thing, as it makes your belief network a lot more complicated, and I'm not sure whether this thing should display numerical probabilities at all. Instead of having to juggle the hypotheses of "Alice lied" and "Bob exaggerates things", the game should probably just record the fact that "Bob exaggerates things". But I spent a bunch of time making these pictures, and they do illustrate some of the general principles involved, so I'll just use them for now.
Game basics
So, to repeat the basic premise of the game, in slightly more words this time around: your task is to figure out something, and in order to do so, you need to collect different pieces of evidence. As you do so, the game generates a belief network showing the origin and history of the various pieces of evidence that you've gathered. That much is done automatically. But often, the evidence that you've gathered is compatible with many different hypotheses. In those situations, you can experiment with different ways of various hypotheses being true or false, and the game will automatically propagate the consequences of that hypothetical through your belief network, helping you decide what angle you should explore next.
Of course, people don't always remember the source of their knowledge, or they might just appeal to personal experiences. Or they might lie about the sources, though that will only happen at the more advanced levels.
As you proceed in the game, you will also be given access to more advanced tools that you can use for making hypothetical manipulations to the belief network. For example, it may happen that many different characters say that armies of vampire bats tend to move about at full moon. Since you hear that information from many different sources, it seems reliable. But then you find out that they all heard it from a nature documentary on TV that aired a few weeks back. This is reflected in your belief graph, as the game modifies it to show that all of those supposedly independent sources can actually be tracked back to a single one. That considerably reduces the reliability of the information.
But maybe you were already suspecting that the sources might not be independent? In that case, it would have been nice if the belief graph interface would let you postulate this beforehand, and see how big of an effect it would make on the plausibility of the different hypotheses if they were in fact reliant on each other. Once your character learns the right skills, it becomes possible to also add new hypothetical connections to the belief graph, and see how this would influence your beliefs. That will further help you decide what possibilities to explore and verify.
Because you can't explore every possible eventuality. There's a time limit: after a certain amount of moves, a bomb will go off, the aliens will invade, or whatever.
The various characters are also more nuanced than just "reliable" or "not reliable". As you collect information about the various characters, you'll figure out their mindware, motivations, and biases. Somebody might be really reliable most of the time, but have strong biases when it comes to politics, for example. Others are out to defame others, or invent fancy details to all the stories. If you talk to somebody you don't have any knowledge about yet, you can set a prior on the extent that you rely on their information, based on your experiences with other people.
You also have another source of evidence: your own intuitions and experience. As you get into various situations, a source of evidence that's labeled simply "your brain" will provide various gut feelings and impressions about things. The claim that Alice presented doesn't seem to make sense. Bob feels reliable. You could persuade Carol to help you if you just said this one thing. But in what situations, and for what things, can you rely on your own brain? What are your own biases and problems? If you have a strong sense of having heard something at some point, but can't remember where it was, are you any more reliable than anyone else who can't remember the source of their information? You'll need to figure all of that out.
As the game progresses to higher levels, your own efforts will prove insufficient for analyzing all the necessary information. You'll have to recruit a group of reliable allies, who you can trust to analyze some of the information on their own and report the results to you accurately. Of course, in order to make better decisions, they'll need you to tell them your conclusions as well. Be sure not to report as true things that you aren't really sure about, or they will end up drawing the wrong conclusions and focusing on the wrong possibilities. But you do need to condense your report somewhat: you can't just communicate your entire belief network to them.
Hopefully, all of this should lead to player learning on a gut level things like:
Design considerations
It's not enough for the game to be educational: if somebody downloads the game because it teaches rationality skills, that's great, but we want people to also play it because it's fun. Some principles that help ensure that, as well as its general utility as an educational aid, include:
What next?
What you've just read is a very high-level design, and a quite incomplete one at that: I've spoken on the need to have "an emotionally compelling story", but said nothing about the story or the setting. This should probably be something like a spy or detective story, because that's thematically appropriate for a game which is about managing information; and it might be best to have it in a fantasy setting, so that you can question the widely-accepted truths of that setting without needing to get on anyone's toes by questioning widely-accepted truths of our society.
But there's still a lot of work that remains to be done with regard to things like what exactly does the belief network look like, what kinds of evidence can there be, how does one make all of this actually be fun, and so on. I mentioned the need to have both short- and medium-term feedback, but I'm not sure of how that could be achieved, or whether this design lets you achieve it at all. And I don't even know whether the game should show explicit probabilities.
And having a design isn't enough: the whole thing needs to be implemented as well, preferably while it's still being designed in order to take advantage of agile development techniques. Make a prototype, find some unsuspecting testers, spring it on them, revise. And then there are the graphics and music, things for which I have no competence for working on.
I'll probably be working on this in my spare time - I've been playing with the idea of going to the field of educational games at some point, and want the design and programming experience. If anyone feels like they could and would want to contribute to the project, let me know.
EDIT: Great to see that there's interest! I've created a mailing list for discussing the game. It's probably easiest to have the initial discussion here, and then shift the discussion to the list.