What would justify making the equal case the default is if over the space of all environment more often than not the environment made no difference.

The environments we encounter are very homogeneous compared to the space of possibilities, enough so that it generally won't flip the ordering of (sufficiently different) minds by intelligence/optimization power. There's no plausible (pre-Singularity) environment in which chimps will suddenly have the technological advantage over humans, though they tie us in the case of global extinction.