Consider the "twin sacrificial volunteer's dilemma":[1]
1000 identical agents have the same deterministic decision-making process and the same observations, and can't communicate with each other. They can decide to either do nothing, or pay $1 to volunteer. If at least one agent volunteers, everyone who didn't volunteer gets $1 million[2]. Volunteers never receive anything. All of this is common knowledge.
Payoff matrix for clarity:
You don't volunteer
You volunteer
Nobody else volunteers
$0
-$1
Someone else volunteers
$1M
-$1
Because the agents all make decisions the same way, there are only two possible outcomes (in bold in the table): everybody volunteers (everybody gets -$1) or nobody volunteers (everybody gets $0). Not volunteering is the dominant strategy, and all reasonable decision theories don't volunteer.
(In this situation the agents can't implement a policy like "volunteer with a small chance", since they are deterministic and don't have access to randomness).[3]
Giving agents serial numbers
Now imagine the same situation, but instead of every agent having the same observations, every agent has a serial number from 1-1000 and is told their serial number (and this is common knowledge). This serial number doesn't change anything about the payoffs. I call this the "numbered sacrificial volunteer's dilemma", and here the symmetry is broken so it's possible for some agents to volunteer and others to not. Does this change things? CDT and EDT agents still don't volunteer for almost the same reasons as before. FDT is more complicated. (This scenario is somewhat similar to this one from the UDT1.1 description.)
FDT is most commonly described as action-based, where you intervene on FDT(P, G, x) (where P and G represent the agent's probability distribution and causal graph, and x is the observations). But you can also formalize it by having it find a policy instead: footnote 16 of the FDT paper says that the authors prefer to formalize it as intervening on FDT(P, G) to find the optimal policy, then choosing actions based on the optimal policy. It's not clear to me how action-FDT acts here, but I believe action-FDT would behave the same as CDT here (never volunteering) and treat each agent's decision process as independent because they have different observations. I'll look at only policy-FDT in the rest of this post.
Policy-based FDT agents find the optimal policy before conditioning on their observations (here, their serial number). There are multiple possible outcomes that are equally good in expectation before you know your serial number: "agent #1 volunteers and everyone else doesn't", "agent #2 volunteers and everyone else doesn't", etc.
FDT (in the paper at least) doesn't specify how you break ties between equally optimal policies here.[4] But since all agents implement the same deterministic decision-making process, they will all break the tie the same way and choose the same designated volunteer. So a specific designated FDT agent will volunteer and the rest won't, which is the optimal outcome.
It's important to note that this relies on the fact that all of the FDT agents use the same decision-making process, and so break the tie in the same way. If the FDT agents broke the tie in different ways then each agent would decide who the designated volunteer agent is differently, which would lead to suboptimal outcomes.
FDT doesn't care about fairness
Note that FDT doesn't care about fairness here - all that matters is the total allocation of utility across the agent-serial numbers you could be, regardless of how fair it is to an individual agent. An agent behind the veil of ignorance (doesn't know their serial number) would prefer that all agents were FDT agents because it's the best (when you average across possible serial numbers).
This scenario is like a counterfactual mugging, where it makes sense for the designated volunteer to pay because they would benefit if they hadn't happened to be the designated volunteer.
Not pre-committing to FDT if you know your number
FDT agents do well in the numbered sacrificial volunteer's dilemma. But agents that know their serial number might not want to precommit to being one: once you know your serial number (or any information that FDT would use to tie-break), you wouldn't want to precommit to being an FDT agent if it would cause tie-breaks to be resolved against you (for the same reason that in the counterfactual mugging, a CDT agent wouldn't want to become an FDT agent after the coin has already landed).
Conclusion
Agents who implement (policy-based) FDT the same way coordinate in the numbered sacrificial volunteer's dilemma, unlike CDT/EDT agents. I think this is an interesting result. And for FDT agents to cooperate in certain situations, they have to implement FDT the same way so that they can break ties the same way when there are multiple equally good sets of actions they could all take.
I'm using dollars here, but really I mean the amount of utility you get (and each agent only cares about their own utility). If the agents aren't risk-neutral, treat "$1M" as "the amount of dollars that gives one million times the utility as $1".
If they had access to randomness, FDT agents would randomly volunteer with some small chance. But the main point of this article is in the next section, where FDT agents can do better than randomness.
In policy-based FDT the policy is a function of just the probability distribution and causal graph; I assume the tie-breaking is inside the policy-selection and so isn't a function of the observations (which would lead to asymmetry).
I've been thinking of FDT as heavily relying on the fact that FDT does things for reasons, which is going to make tie-breaking rather tricky. Are we sure a good decision theory needs to be able to choose a particular direction into which to fall over, instead of deciding to throw a coin about it?
Consider the "twin sacrificial volunteer's dilemma":[1]
Payoff matrix for clarity:
You don't volunteer
You volunteer
Nobody else volunteers
$0
-$1
Someone else volunteers
$1M
-$1
Because the agents all make decisions the same way, there are only two possible outcomes (in bold in the table): everybody volunteers (everybody gets -$1) or nobody volunteers (everybody gets $0). Not volunteering is the dominant strategy, and all reasonable decision theories don't volunteer.
(In this situation the agents can't implement a policy like "volunteer with a small chance", since they are deterministic and don't have access to randomness).[3]
Giving agents serial numbers
Now imagine the same situation, but instead of every agent having the same observations, every agent has a serial number from 1-1000 and is told their serial number (and this is common knowledge). This serial number doesn't change anything about the payoffs. I call this the "numbered sacrificial volunteer's dilemma", and here the symmetry is broken so it's possible for some agents to volunteer and others to not. Does this change things? CDT and EDT agents still don't volunteer for almost the same reasons as before. FDT is more complicated. (This scenario is somewhat similar to this one from the UDT1.1 description.)
FDT is most commonly described as action-based, where you intervene on FDT(P, G, x) (where P and G represent the agent's probability distribution and causal graph, and x is the observations). But you can also formalize it by having it find a policy instead: footnote 16 of the FDT paper says that the authors prefer to formalize it as intervening on FDT(P, G) to find the optimal policy, then choosing actions based on the optimal policy. It's not clear to me how action-FDT acts here, but I believe action-FDT would behave the same as CDT here (never volunteering) and treat each agent's decision process as independent because they have different observations. I'll look at only policy-FDT in the rest of this post.
Policy-based FDT agents find the optimal policy before conditioning on their observations (here, their serial number). There are multiple possible outcomes that are equally good in expectation before you know your serial number: "agent #1 volunteers and everyone else doesn't", "agent #2 volunteers and everyone else doesn't", etc.
FDT (in the paper at least) doesn't specify how you break ties between equally optimal policies here.[4] But since all agents implement the same deterministic decision-making process, they will all break the tie the same way and choose the same designated volunteer. So a specific designated FDT agent will volunteer and the rest won't, which is the optimal outcome.
It's important to note that this relies on the fact that all of the FDT agents use the same decision-making process, and so break the tie in the same way. If the FDT agents broke the tie in different ways then each agent would decide who the designated volunteer agent is differently, which would lead to suboptimal outcomes.
FDT doesn't care about fairness
Note that FDT doesn't care about fairness here - all that matters is the total allocation of utility across the agent-serial numbers you could be, regardless of how fair it is to an individual agent. An agent behind the veil of ignorance (doesn't know their serial number) would prefer that all agents were FDT agents because it's the best (when you average across possible serial numbers).
This scenario is like a counterfactual mugging, where it makes sense for the designated volunteer to pay because they would benefit if they hadn't happened to be the designated volunteer.
Not pre-committing to FDT if you know your number
FDT agents do well in the numbered sacrificial volunteer's dilemma. But agents that know their serial number might not want to precommit to being one: once you know your serial number (or any information that FDT would use to tie-break), you wouldn't want to precommit to being an FDT agent if it would cause tie-breaks to be resolved against you (for the same reason that in the counterfactual mugging, a CDT agent wouldn't want to become an FDT agent after the coin has already landed).
Conclusion
Agents who implement (policy-based) FDT the same way coordinate in the numbered sacrificial volunteer's dilemma, unlike CDT/EDT agents. I think this is an interesting result. And for FDT agents to cooperate in certain situations, they have to implement FDT the same way so that they can break ties the same way when there are multiple equally good sets of actions they could all take.
Which is the volunteer's dilemma, but where the volunteer doesn't benefit and the agents are identical.
I'm using dollars here, but really I mean the amount of utility you get (and each agent only cares about their own utility). If the agents aren't risk-neutral, treat "$1M" as "the amount of dollars that gives one million times the utility as $1".
If they had access to randomness, FDT agents would randomly volunteer with some small chance. But the main point of this article is in the next section, where FDT agents can do better than randomness.
In policy-based FDT the policy is a function of just the probability distribution and causal graph; I assume the tie-breaking is inside the policy-selection and so isn't a function of the observations (which would lead to asymmetry).