Suddenly I find myself confused. Why is this acausal?
Suppose I buy shares in a company that builds an AI, which then works for the good of the company, which rewards share-owners. This is ordinary causality: I contributed towards its building, and was rewarded later.
Suppose I contribute towards something other than its building, in the belief that an AI which will later come into being will reward me for having done this. Still doesn't seem acausal to me.
Suppose I believe an AI is likely to be built that will conquer the world and transfer all wealth to its builders. Then I would want to be among the builders. This is ordinary acting-on-expected-value. But those who aren't builders get negative value (~~ are tortured) by the AI if it's built.
Suppose I buy shares in a company that builds an AI, which then works for the good of the company, which rewards share-owners. This is ordinary causality: I contributed towards its building, and was rewarded later.
What makes it possible to be rewarded as a shareholder is a legal system which enforces your ownership rights: a kind of pre-commitment which is feasible even among humans who cannot show proofs about their "source code." The legal system is a mutual enforcement system which sets up a chain of causality towards your being paid back....
Todays xkcd
I guess there'll be a fair bit of traffic coming from people looking it up?