You can practice forecasting on videogames you've never played before, to develop the muscles for "decision-relevant forecasting."
Turn based videogames work best. I recommend "Luck Be a Landlord", "Battle for Polytopia", or "Into the Breach."
Each turn, make as many Fatebook predictions as you can in 5 minutes, then actually make your decision(s) for the turn.
After 3 turns, instead of making "as many predictions as possible", switch to trying to come up with at least two mutually exclusive actions you might take this turn, and come up with predictions that would inform which action to take.
Don't forget to follow this up with practicing forecasting for decisions you're making in "real life", to improve transfer learning. And, watch out for accidentally just getting yourself addicted to videogames, if you weren't already in that boat.
This is pretty fun to do in groups and makes for a good meetup, if you're into that.
Recently I published Exercise: Planmaking, Surprise Anticipation, and "Baba is You". In that exercise, you try to make a complete plan for solving a puzzle-level in a videogame, without interacting with the world (on levels where you don't know what all the objects in the environment do), and solve it on your first try.
Several people reported it pretty valuable (it was highest rated activity at my metastrategy workshop). But, it's fairly complicated as an exercise, and a single run of the exercise typically takes at least an hour (and maybe several hours) before you get feedback on whether you're "doing it right."
It'd be nice to have a way to practice decision-relevant forecasting with a faster feedback loop. I've been exploring the space of games that are interesting to "one-shot". (i.e. " try to win on your first playthrough"), and also exploring the space of exercises that take advantage of your first playthrough of a game.
So, an additional, much simpler exercise that I also like, is:
Play a turn-based game you haven't played before.
Each turn, set a 5 minute timer for making as many predictions as you can about how the game works, what new rules or considerations you might learn later. Then, a 1 minute timer for actually making your choices for what action(s) to take during the turn.
And... that's it. (to start with, anyway).
Rather that particularly focusing on "trying really hard to win", start with just making lots of predictions, about a situation where you're at least trying to win a little, so you can develop the muscles of noticing what sort of predictions you can make while you're in the process of strategically orienting. And, notice what sorts of implicit knowledge you have, even though you don't technically "know" how the game would work.
Some of the predictions might resolve the very next turn. Some might resolve before the very next turn, depending on how many choices you get each turn. And, some might take a few turns, or even pretty deep into the game. Making a mix of forecasts of different resolution-times is encouraged.
I think there are a lot of interesting skills you can layer on top of this, after you've gotten the basic rhythm of it. But "just make a ton of predictions about a domain where you're trying to achieve something, and get quick feedback on it" seems like a good start.
Choosing Games
Not all games are a good fit for this exercise. I've found a few specific games I tend to recommend, and some principles for which games to pick.
The ideal game has:
Minimal (or skippable) tutorial. A major point of the exercise is to make prediction about the game mechanics and features. Good games for this exercise a) don't spoonfeed you all the information about the game, but also b) are self-explanatory enough to figure out without a tutorial.
Turn based (or, with natural stopping points before decisions). This isn't a hard rule, but this exercise is meant to build up your mental muscles for practical forecasting on longterm goals. You can practice it on twitchy reflex games but it's adding an extra layer of difficulty without much benefit.
Relatively deep strategy of some kind. The game should reward thinking ahead about what future choices you're likely to have. Most of my exploration here has been on strategy games and puzzle games.
Information you don't initially know (but which is reasonably predictable). Some strategy games frontload all the information on you, and then this is just an exercise in reading every single menu and tooltip. This might also be useful for rationality practice, but it's not the point of this exercise. Ideal games here give you enough information at a time to chew on, but not overwhelm you. And, they leave room to make predictions in the future.
Short playtime. You don't have to complete a whole game in order to get value out of this exercise, but it's useful if the game is short, such that you get to make predictions about how the midgame and endgame might play out, and then get results within an hour or so.
One question is "what counts as a 'turn?'", which varies a bit from game to game. Some games allow basically one action per turn. Some have multiple possible actions within a turn (which might be irreversible, and resolve a prediction you just made).
Unfortunately, it's not always easy to tell what games will satisfy the above criteria, but you can kinda get a feel for it.
Some recommended games
Luck be a landlord is a very simple game, which makes for a good streamlined experience. I especially recommend it for people new to either gaming or forecasting. At the very beginning of the game, I recommend making at least one prediction per button click, and then within a minute or so you'll have probably figured out "what is a turn?", and can continue with the "5 minutes per turn, generate as many predictions as you can" rule.
The Battle of Polytopia is a more complex strategy game. One caveat: by default the game will start you out with a tutorial, defeating the exercise. But, you can cancel this tutorial and return to the main menu, and start a new game. (I recommend setting the difficulty to something that will be a challenge. I suggest giving yourself 3 opponents, on a "hard" difficulty).
Into the Breachisn't quite as good as the previous few games (the game becomes more predictable sooner). But if you've already played the first two it's still a good option. Note that it has a somewhat longer "opening story/context" section. I recommend starting the exercise after "your three dudes drop from the dropship."
How to make predictions
As always, I recommend Fatebook.io as a good tool for quickly making lightweight predictions, while having useful integrations that also make it a good longterm power tool. (See my writeup in Fluent, Cruxy Predictions for how to use Fatebook more generally)
If you've installed the Fatebook chrome/firefox extension, you can make predictions in a google-doc or similar while also taking general notes about the game. (Warning: doesn't work with adblock, which by default includes browsers like Brave.)
If you don't like Fatebook, I recommend writing them somewhere that makes it easy to followup and grade the predictions and see your calibration curve.. I think there's a significant differnence between simply making the prediction, and forcing yourself to grade it and starting to establish what your calibration curve looks like.
Phase 2: Decision-Relevant Predictions
After you've gotten a handle on the basics of "make as many predictions as you can", I recommend layering on "specifically make predictions that help inform your decisions."
After a few turns (typically I recommend 4-5 turns, but, depends on your personal experience), each turn, try to think of 2-3 mutually exclusive decisions that seem plausibly like "the right move." Figure out which move seems best to you.
Then, try to come up with an prediction that would change your mind about which move is best. The prediction can be about concrete facts about the game, or updates you might make to your strategic frame.
Some example predictions (phrased as concrete statements, which you can assign a probability to)
"A particular resource is will be my bottleneck in a few turns."
"The game will introduce [some particular new element]."
"At the end of the game, I'll think it was useful to think of [X] as my intermediate goal, at the current stage of the game."
Sometimes, you'll be finding a crux between your favorite plan and your second-favorite plan. Sometimes, you'll end up deciding between the first and third-best-seeming plan, because while it seems worse under your mainline assumptions, it's more likely for you to end up getting surprised in a way that changes your whole strategic frame than to discover the second-best option is better than your favorite.
Interlude: Metastrategy Brainstorming
After 2-3 rounds of Cruxy Predicting, I recommend setting a 7 minute timer to explicit ask yourself:
"What actually am I trying to do with this exercise? And, how could I do it better?". Brainstorm as many strategies or considerations that might help you get more value out of the exercise.
If you're running this as a meetup, afterward the brainstorming, it's nice for people to share their ideas and discuss it.
the skill of "what am I trying to do and how can I do it better" is a generally important skill that should interweave with all rationality training
it shifts you from a mindset of "I'm trying to do this exercise without quite understanding it" to being a more of an agent who's trying to do something on purpose, which makes you more likely to actually learn and internalize useful things.
It makes it more likely for you to "adjust your seat", adapting the exercise to whatever is most helpful for you
my experience is that with 7 minutes, I just concretely generate at least 1 strategy I hadn't previously been considering that is decently helpful.
After your brainstorming, continue for another few rounds of cruxy prediction and see if you notice any changes. And, eventually, try the exercise with a different game and see if it plays differently.
"Okay I basically get it" vs "Predict how masters think."
The first few rounds of a new game have a lot of surprise factor. Every few minutes a new element is introduced.
Frequently, once people get midway into a game, they feel like "okay, I get it", and they find themselves shifting into a mode where they are more like "just playing the game" than doing any deliberate rationality exercise.
I'm not sure what to think about this. I feel this impulse myself. But, I notice that there are tons of skills, concepts and frameworks that experts at a game tend to use, that I wouldn't have thought about on my own.
I haven't actually had success with this yet. (I played a bunch of Slay the Spire without having looked up what Jorbs had to say about it, and failed to actually figure out higher-level principles on my own before accidentally reading some bits about them on the internet from other people).
But, I still feel hopeful about "predict what concepts a master would use" as lens worth thinking about. Once you feel like "you get it" for the basic concepts of a game, I recommend asking yourself "what skills, concepts or frameworks do I predict a master would employ here?".
Finally: Remember to connect this with "Real Life" practice
Some people ask me "So, do you think getting good at predicting strategy games automatically makes you good at winning at life? That seems sketchy to me. Most people good at games don't seem to be sitting on top of a heap of utility."
What I think does work is following up your Toy Exercises with real life practice. Look at your upcoming week and make some predictions about it. See if you can come up with two mutually exclusive ways of spending your time, and make predictions about what would change your mind about which one to prioritize.
Actively ask yourself, which of the mental motions that you learned while practicing on Quick Feedbackloop video games apply to your real life.
I'm using "decision-relevant" and "cruxy" as roughly interchangeable. I've noticed that "decision-relevant" is a more accessible phrase, although I think "crux" and "cruxy" are useful concepts to have a short handle for and worth introducing to people.
Cliff notes:
Don't forget to follow this up with practicing forecasting for decisions you're making in "real life", to improve transfer learning. And, watch out for accidentally just getting yourself addicted to videogames, if you weren't already in that boat.
This is pretty fun to do in groups and makes for a good meetup, if you're into that.
Recently I published Exercise: Planmaking, Surprise Anticipation, and "Baba is You". In that exercise, you try to make a complete plan for solving a puzzle-level in a videogame, without interacting with the world (on levels where you don't know what all the objects in the environment do), and solve it on your first try.
Several people reported it pretty valuable (it was highest rated activity at my metastrategy workshop). But, it's fairly complicated as an exercise, and a single run of the exercise typically takes at least an hour (and maybe several hours) before you get feedback on whether you're "doing it right."
It'd be nice to have a way to practice decision-relevant forecasting with a faster feedback loop. I've been exploring the space of games that are interesting to "one-shot". (i.e. " try to win on your first playthrough"), and also exploring the space of exercises that take advantage of your first playthrough of a game.
So, an additional, much simpler exercise that I also like, is:
And... that's it. (to start with, anyway).
Rather that particularly focusing on "trying really hard to win", start with just making lots of predictions, about a situation where you're at least trying to win a little, so you can develop the muscles of noticing what sort of predictions you can make while you're in the process of strategically orienting. And, notice what sorts of implicit knowledge you have, even though you don't technically "know" how the game would work.
Some of the predictions might resolve the very next turn. Some might resolve before the very next turn, depending on how many choices you get each turn. And, some might take a few turns, or even pretty deep into the game. Making a mix of forecasts of different resolution-times is encouraged.
I think there are a lot of interesting skills you can layer on top of this, after you've gotten the basic rhythm of it. But "just make a ton of predictions about a domain where you're trying to achieve something, and get quick feedback on it" seems like a good start.
Choosing Games
Not all games are a good fit for this exercise. I've found a few specific games I tend to recommend, and some principles for which games to pick.
The ideal game has:
One question is "what counts as a 'turn?'", which varies a bit from game to game. Some games allow basically one action per turn. Some have multiple possible actions within a turn (which might be irreversible, and resolve a prediction you just made).
Unfortunately, it's not always easy to tell what games will satisfy the above criteria, but you can kinda get a feel for it.
Some recommended games
Luck be a landlord is a very simple game, which makes for a good streamlined experience. I especially recommend it for people new to either gaming or forecasting. At the very beginning of the game, I recommend making at least one prediction per button click, and then within a minute or so you'll have probably figured out "what is a turn?", and can continue with the "5 minutes per turn, generate as many predictions as you can" rule.
The Battle of Polytopia is a more complex strategy game. One caveat: by default the game will start you out with a tutorial, defeating the exercise. But, you can cancel this tutorial and return to the main menu, and start a new game. (I recommend setting the difficulty to something that will be a challenge. I suggest giving yourself 3 opponents, on a "hard" difficulty).
Into the Breach isn't quite as good as the previous few games (the game becomes more predictable sooner). But if you've already played the first two it's still a good option. Note that it has a somewhat longer "opening story/context" section. I recommend starting the exercise after "your three dudes drop from the dropship."
How to make predictions
As always, I recommend Fatebook.io as a good tool for quickly making lightweight predictions, while having useful integrations that also make it a good longterm power tool. (See my writeup in Fluent, Cruxy Predictions for how to use Fatebook more generally)
If you've installed the Fatebook chrome/firefox extension, you can make predictions in a google-doc or similar while also taking general notes about the game. (Warning: doesn't work with adblock, which by default includes browsers like Brave.)
If you don't like Fatebook, I recommend writing them somewhere that makes it easy to followup and grade the predictions and see your calibration curve.. I think there's a significant differnence between simply making the prediction, and forcing yourself to grade it and starting to establish what your calibration curve looks like.
Phase 2: Decision-Relevant Predictions
After you've gotten a handle on the basics of "make as many predictions as you can", I recommend layering on "specifically make predictions that help inform your decisions."
Ultimately, the goal is to make fluent, cruxy[1] predictions about choices that matter to you.
After a few turns (typically I recommend 4-5 turns, but, depends on your personal experience), each turn, try to think of 2-3 mutually exclusive decisions that seem plausibly like "the right move." Figure out which move seems best to you.
Then, try to come up with an prediction that would change your mind about which move is best. The prediction can be about concrete facts about the game, or updates you might make to your strategic frame.
Some example predictions (phrased as concrete statements, which you can assign a probability to)
Sometimes, you'll be finding a crux between your favorite plan and your second-favorite plan. Sometimes, you'll end up deciding between the first and third-best-seeming plan, because while it seems worse under your mainline assumptions, it's more likely for you to end up getting surprised in a way that changes your whole strategic frame than to discover the second-best option is better than your favorite.
Interlude: Metastrategy Brainstorming
After 2-3 rounds of Cruxy Predicting, I recommend setting a 7 minute timer to explicit ask yourself:
"What actually am I trying to do with this exercise? And, how could I do it better?". Brainstorm as many strategies or considerations that might help you get more value out of the exercise.
If you're running this as a meetup, afterward the brainstorming, it's nice for people to share their ideas and discuss it.
See "Metastrategic Brainstorming" for more detail here. I think this is important because:
After your brainstorming, continue for another few rounds of cruxy prediction and see if you notice any changes. And, eventually, try the exercise with a different game and see if it plays differently.
"Okay I basically get it" vs "Predict how masters think."
The first few rounds of a new game have a lot of surprise factor. Every few minutes a new element is introduced.
Frequently, once people get midway into a game, they feel like "okay, I get it", and they find themselves shifting into a mode where they are more like "just playing the game" than doing any deliberate rationality exercise.
I'm not sure what to think about this. I feel this impulse myself. But, I notice that there are tons of skills, concepts and frameworks that experts at a game tend to use, that I wouldn't have thought about on my own.
I haven't actually had success with this yet. (I played a bunch of Slay the Spire without having looked up what Jorbs had to say about it, and failed to actually figure out higher-level principles on my own before accidentally reading some bits about them on the internet from other people).
But, I still feel hopeful about "predict what concepts a master would use" as lens worth thinking about. Once you feel like "you get it" for the basic concepts of a game, I recommend asking yourself "what skills, concepts or frameworks do I predict a master would employ here?".
Finally: Remember to connect this with "Real Life" practice
Some people ask me "So, do you think getting good at predicting strategy games automatically makes you good at winning at life? That seems sketchy to me. Most people good at games don't seem to be sitting on top of a heap of utility."
To which I say: "No, not automatically. It seems like transfer learning is pretty difficult. Tons of psychologists have tried to discover metalearning or transfer learning over a hundred years and failed, despite being very biased in favor of thinking it should be possible. I... actually do think those hundred years worth of psychologists did a bunch of dumb things and think my ideas make more sense. But, it's a pretty cautionary tale.
What I think does work is following up your Toy Exercises with real life practice. Look at your upcoming week and make some predictions about it. See if you can come up with two mutually exclusive ways of spending your time, and make predictions about what would change your mind about which one to prioritize.
Actively ask yourself, which of the mental motions that you learned while practicing on Quick Feedbackloop video games apply to your real life.
I'm using "decision-relevant" and "cruxy" as roughly interchangeable. I've noticed that "decision-relevant" is a more accessible phrase, although I think "crux" and "cruxy" are useful concepts to have a short handle for and worth introducing to people.
I'm splitting the difference by using both here.