If you're using average utility, then whether this discovery has been made affects whether you want to create that other universe. Is that correct?

With the standard caveats, yes that seems reasonable. Given the existence of that ultrahappy universe an average human life will be more likely to exist in happier circumstances than the ones in the multiversal reality I'd create if I chose to add that less-than-averagely-happy universe.

Same way as I'd not take 20% of actual existing happy people and force them to live less happy lives.

Think about all sentient lives as if they were part of a single mind, called "Sentience". We design portions of Sentience's life. We want as much a proportion of Sentience's existence to be as happy as possible, satisfying Sentience's preferences.