What I had in mind was that each of the AIs would come up with a distribution over the kinds of civilizations which are likely to arise in the universe by predicting the kinds of planets out there (which is presumably something you can do since even we have models for this) and figuring out different potential evolutions for life that arises on those planets. Does that make sense?
I was going to respond saying I didn't think that would work as a method, but now I'm not so sure.
My counterargument would be to suggest that there's no goal system which can't arbitrarily come about as a Fisherian Runaway, and that our AI's acausal trade partners could be working on pretty much any optimisation criteria whatsoever. Thinking about it a bit more, I'm not entirely sure the Fisherian Runaway argument is all that robust. There is, for example, presumably no Fisherian Runaway goal of immediate self-annihilation.
If there's some sort of structu...
If it's worth saying, but not worth its own post, even in Discussion, it goes here.