I think you may be missing PhilGoetz's point.

E(f) needn't have anything to do with W. But it has to do with the f-values of all the different versions of you that there might be in the future (or in different Everett branches, or whatever). And it treats all of them symmetrically, looking only at their average and not at their distribution.

That is: the way in which you decide what to do can be expressed in terms of some preference you have about the state of the world, viewed from the perspective of one possible-future-you, but all those possible-future-yous have to be treated symmetrically, just averaging their preferences. (By "preference" I just mean anything that plays the role of f.)

So, Phil says, if that's the only consistent way to treat all the different versions of you, then surely it's also the only consistent way to treat all the different people in the world.

(This is of course the controversial bit. It's far from obvious that you should see the possible-future-yous in the same way as you see the actual-other-people. For instance, because it's more credible to think of those possible-future-yous as having the same utility function as one another. And because we all tend to care more about the welfare of our future selves than we do about other people's. And so on.)

If so, then the following would be true: To act consistently, your actions must be such as to maximize the average of something (which, yes, need have nothing to do with the functions w, but it had better in some sense be the same something) over all actual and possible people.

I think Phil is wrong, but your criticisms don't seem fair to me.