If, in a multiverse, I know with certainty the objective frequency distribution of single-world outcomes, then yes, I just pick the action with the highest utility.

You still need to multiply utility of those events by their probability. Objective frequency is not as objective as it may seem, it's just a point at which the posterior is no longer expected to change, given more information. Or, alternatively, it's "physical probability", a parameter in your model that has nothing to do with subjective probability and expected utility maximization, and has the status similar to that of, say, mass.