Ever since I read Scott Alexander’s “The AI superforecasters are here,” I have been excited by the possibility of experimenting with using automated superforecasting for decision-making, or as I wrote about previously, automated decision forecasting/forecasters (ADF[1]). Decision forecasting is already an advertised option on the front-page of FutureSearch.
The premise of automated decision forecasting is that when a user faces a decision, they would tell the ADF their overall goal and the options they are considering. The ADF would then forecast how likely the user is to achieve their goal if they choose each option.
For example, a user would input their desired goal and options, “I want to be hired for a role in AI safety by March 1st 2027”, and “I am deciding between applying to x, y, and z fellowship.”
The ADF would then think through everything it knows about these fellowships, acceptance and job placement base rates, you and your history, etc., and give a quantitative estimate for how likely each fellowship will lead to a role in AI safety.
The user may give as much relevant background information as is helpful, such as résumé or work history; optionally the ADF can ask for any crucial information, or the user could have a structured repository of personal data saved if they use the system often.
Perhaps most exciting, for the most important decisions, the ADF could automatically generate tens or hundreds of options based on the user’s high level goals. It could ask the user targeted questions to help it efficiently narrow down to options the user will find most appealing and effective. For the top several options it could then decision forecast the effect of each option for achieving the user’s goals.
The user could also use the ADF system for smaller scale decisions such as which aspect of their project to work on first, who to reach out to for feedback or collaboration, what media or research to read, etc.
Or the user could keep a daily log of their progress and the ADF could automatically generate numerous recommendations every day for the user to consider based on the user’s intermediate goals.
The system could save and periodically update a map of intermediate goals[2] which it predicts are most likely to achieve higher level goals, backchaining from the terminal goal all the way down to daily decisions.
You could also give the ADF feedback over time, both on your achievement of various intermediate and high-level goals, and on your feelings about its recommendations, and it could use this to personalize its forecasts and recommendations.
One concern is that using a single high-level goal may lead to hyper-focus to the exclusion of other personal and professional factors. To address this, the ADF system could hold multiple high-level goals to reduce myopic focus, and for each decision separately forecast each option’s impact on each high-level goal, then use some procedure to aggregate these, allowing the user to simultaneously balance the pursuit of several goals. Unfortunately, each high-level goal added in this way would increase cost linearly, assuming each is forecast independently. Two forecasts of mine in FutureSearch cost about $10 combined, so having many high-level goals, many options for each decision, and many decisions could quickly become prohibitively expensive. Hence, until AI prices fall, or clever methods help reduce prices, it may be necessary to focus on only the most important high-level decisions.
A major concern is whether current AI is capable of forecasting personal conditional forecasts at all, as current AI superforecasting has focused mainly on public events with unconditional forecasts.
My guess is that the forecasting process of personal vs. public events is similar enough that there will be some transfer (though tailored personal forecasting systems would likely be much better), but I’m less confident about conditional forecasting due to weird potential failure modes. (Edit: I wrote this on a flight, and when I checked FutureSearch for the first time in several months, it looks like they already have decision forecasting advertised on their front-page! But I think it’s still an open question how effective these systems are at decision forecasting.)
Another place this could break down is zooming out farther to long-term/large-scale forecasts where there is little base-rate data, and especially mixing this with short-term conditions; e.g. “if I choose fellowship A vs fellowship B, how will that affect the eventual chance of AI takeover.” I feel a lot less confident about such a system working on forecasts that are at this high level, which is unfortunate, as another mechanism that I think would be really interesting is running a huge number of conditional forecasts at a high level, and perhaps even run a contest, to see which interventions or intermediate goals are most likely to achieve our high level goals, for example:
“How much would a $1,000,000 donation to each of the following increase the chances of humans retaining control of AI by 2100: A. Lobbying for a pause B. An organization that develops verification tech requred for a slow-down C. Bengio’s 'Scientist AI' Etc. etc. etc. across several hundred interventions”
Again, by broadly forecasting many such conditionals across many high-level goals, we could create a heat-map of which interventions are most likely to achieve certain high-level goals or intermediate goals, and which intermediate goals are most likely to help with various high-level goals, and to what degree. While some high level goals may be out-of-reach, perhaps intermediate goals are more realistic to forecast, and insofar as certain intermediate goals seem to robustly increase the chances of many good high-level outcomes, perhaps this would mean we have found some of the “societal primary goods” MacAskill describes in “Viatopia”. It seems important here to consider both negative and positive goals, things we want and don’t want, such as avoiding existential risks or risk factors, and increasing factors that help with good decision making, reflective public institutions, broadly increasing wellbeing and wisdom, and moral progress.
This points to another possibility: ADF could be used to help not just individuals, but organizations to achieve their goals more effectively. It is not hard to imagine that ADF could become a standard process for organizations making big decisions, and eventually small[3] ones as well. And of course, ADF may be a key component of fully autonomous organizations.
If ADF helps individuals and organizations be significantly more effective at achieving their goals, then it’s my sincere hope that ambitious altruistic and rational actors are some of the first to adopt such tools.
It’s concerning that such tools might scale in effectiveness with compute, and so perhaps it is ultimately those with the greatest financial resources who will benefit most. Still, I think this could take a long time to catch on, and indeed many in the EA and rationalist communities will be early adopters. Yet I worry we may not invest heavily enough. It seems to me that, considering substantial funding may be coming to the ecosystem soon, we have been far too slow to scale up capital, and experimentation on high-upside projects like this could be well worth the expenditure.
And perhaps other user-based metrics like financial resources, time constraints; and fuzzy metrics like personal motivation ratings, wellbeing ratings, and social relationship effectiveness across various relationships, etc.
Ever since I read Scott Alexander’s “The AI superforecasters are here,” I have been excited by the possibility of experimenting with using automated superforecasting for decision-making, or as I wrote about previously, automated decision forecasting/forecasters (ADF[1]). Decision forecasting is already an advertised option on the front-page of FutureSearch.
The premise of automated decision forecasting is that when a user faces a decision, they would tell the ADF their overall goal and the options they are considering. The ADF would then forecast how likely the user is to achieve their goal if they choose each option.
For example, a user would input their desired goal and options, “I want to be hired for a role in AI safety by March 1st 2027”, and “I am deciding between applying to x, y, and z fellowship.”
The ADF would then think through everything it knows about these fellowships, acceptance and job placement base rates, you and your history, etc., and give a quantitative estimate for how likely each fellowship will lead to a role in AI safety.
The user may give as much relevant background information as is helpful, such as résumé or work history; optionally the ADF can ask for any crucial information, or the user could have a structured repository of personal data saved if they use the system often.
Perhaps most exciting, for the most important decisions, the ADF could automatically generate tens or hundreds of options based on the user’s high level goals. It could ask the user targeted questions to help it efficiently narrow down to options the user will find most appealing and effective. For the top several options it could then decision forecast the effect of each option for achieving the user’s goals.
The user could also use the ADF system for smaller scale decisions such as which aspect of their project to work on first, who to reach out to for feedback or collaboration, what media or research to read, etc.
Or the user could keep a daily log of their progress and the ADF could automatically generate numerous recommendations every day for the user to consider based on the user’s intermediate goals.
The system could save and periodically update a map of intermediate goals[2] which it predicts are most likely to achieve higher level goals, backchaining from the terminal goal all the way down to daily decisions.
You could also give the ADF feedback over time, both on your achievement of various intermediate and high-level goals, and on your feelings about its recommendations, and it could use this to personalize its forecasts and recommendations.
One concern is that using a single high-level goal may lead to hyper-focus to the exclusion of other personal and professional factors. To address this, the ADF system could hold multiple high-level goals to reduce myopic focus, and for each decision separately forecast each option’s impact on each high-level goal, then use some procedure to aggregate these, allowing the user to simultaneously balance the pursuit of several goals. Unfortunately, each high-level goal added in this way would increase cost linearly, assuming each is forecast independently. Two forecasts of mine in FutureSearch cost about $10 combined, so having many high-level goals, many options for each decision, and many decisions could quickly become prohibitively expensive. Hence, until AI prices fall, or clever methods help reduce prices, it may be necessary to focus on only the most important high-level decisions.
A major concern is whether current AI is capable of forecasting personal conditional forecasts at all, as current AI superforecasting has focused mainly on public events with unconditional forecasts.
My guess is that the forecasting process of personal vs. public events is similar enough that there will be some transfer (though tailored personal forecasting systems would likely be much better), but I’m less confident about conditional forecasting due to weird potential failure modes. (Edit: I wrote this on a flight, and when I checked FutureSearch for the first time in several months, it looks like they already have decision forecasting advertised on their front-page! But I think it’s still an open question how effective these systems are at decision forecasting.)
Another place this could break down is zooming out farther to long-term/large-scale forecasts where there is little base-rate data, and especially mixing this with short-term conditions; e.g. “if I choose fellowship A vs fellowship B, how will that affect the eventual chance of AI takeover.” I feel a lot less confident about such a system working on forecasts that are at this high level, which is unfortunate, as another mechanism that I think would be really interesting is running a huge number of conditional forecasts at a high level, and perhaps even run a contest, to see which interventions or intermediate goals are most likely to achieve our high level goals, for example:
“How much would a $1,000,000 donation to each of the following increase the chances of humans retaining control of AI by 2100:
A. Lobbying for a pause
B. An organization that develops verification tech requred for a slow-down
C. Bengio’s 'Scientist AI'
Etc. etc. etc. across several hundred interventions”
Again, by broadly forecasting many such conditionals across many high-level goals, we could create a heat-map of which interventions are most likely to achieve certain high-level goals or intermediate goals, and which intermediate goals are most likely to help with various high-level goals, and to what degree. While some high level goals may be out-of-reach, perhaps intermediate goals are more realistic to forecast, and insofar as certain intermediate goals seem to robustly increase the chances of many good high-level outcomes, perhaps this would mean we have found some of the “societal primary goods” MacAskill describes in “Viatopia”. It seems important here to consider both negative and positive goals, things we want and don’t want, such as avoiding existential risks or risk factors, and increasing factors that help with good decision making, reflective public institutions, broadly increasing wellbeing and wisdom, and moral progress.
This points to another possibility: ADF could be used to help not just individuals, but organizations to achieve their goals more effectively. It is not hard to imagine that ADF could become a standard process for organizations making big decisions, and eventually small[3] ones as well. And of course, ADF may be a key component of fully autonomous organizations.
If ADF helps individuals and organizations be significantly more effective at achieving their goals, then it’s my sincere hope that ambitious altruistic and rational actors are some of the first to adopt such tools.
It’s concerning that such tools might scale in effectiveness with compute, and so perhaps it is ultimately those with the greatest financial resources who will benefit most. Still, I think this could take a long time to catch on, and indeed many in the EA and rationalist communities will be early adopters. Yet I worry we may not invest heavily enough. It seems to me that, considering substantial funding may be coming to the ecosystem soon, we have been far too slow to scale up capital, and experimentation on high-upside projects like this could be well worth the expenditure.
AI Decision Forecasting/Forecasters could be synonymous and use the same acronym.
And perhaps other user-based metrics like financial resources, time constraints; and fuzzy metrics like personal motivation ratings, wellbeing ratings, and social relationship effectiveness across various relationships, etc.
Obligatory reference to the Whispering Earring.