we can also reasonably know that since we refuse, it doesn't get built in the first place.

The key is that the AI precommits to building it whether we refuse or not.

If we actually do refuse, this precommitment ends up being bad for it, since it builds it without any gain. However, this precommitment, by preventing us from saying "if we refuse, it doesn't get built", also decreases the measure of worlds where it builds it without gaining.