Suppose I get my weights from inside me, and you get yours from inside you; then we might not be able to coordinate, instead wrestling each other over the ability to flip the switch.

In practice people with different values manage to coordinate perfectly fine via trade; I agree an external source of morality would be sufficient for cooperation, but it's not necessary (also having all humans really take an external source as the real basis for all their choices would require some pretty heavy rewriting of human nature).