The "probability" of the imagined world is low, so the opportunity cost of this action makes it wrong. If there was a world fitting your description that had significant "probability" (for example, if you deduced that a past random event turning out differently would likely lead to the situation as you describe it), it would be a plausibly correct action to take.

(The unclear point is what contributes to a world's "probability"; presumably, arbitrary stipulations drive it down, so most thought experiments are morally irrelevant.)