I suspect that the answer to the alien-ball case may be empirical rather than philosophical.

Suppose that there existed quantum configurations in which the alien threw in a red ball, and there existed quantum configurations in which the alien threw in a blue ball, and both of those have approximately equal causal influence on the configuration-cluster in which we are having (approximately) this conversation. In this case, we would happen to be living in a particular type of world such that there was no fact of the matter as to which color ball it was (except that e.g. it mostly wasn't green).