I think aspiring rationalist humans should see it as their responsibility to actually choose one direction in which they want to steer the world, rather than specifying conflicting goals and then asking for some formula that will decide for them how to trade these goals against each other. If you choose to trade off different goals by weighing them with different factors, fine; but if you try to find some 'laws of rationality' that will tell you the one correct way to trade off these goals, without ever needing to make a decision about this yourself, I think you're trying to pass off a responsibility that is properly yours.

Why so much emphasis on "responsibility"? In my mind, I have a responsibility to fulfill any promises I make to others and ... and that's about it. As for figuring out what my preferences are, or should be, I'm going to try any promising approaches I can find, and see if one of them works out. Thinking of myself as a bunch of sub-agents and using ideas from bargaining theory is one such an approach. Trying to solve normative ethics using the methods of moral philosophers may be another. When you say "see it as their responsibility to actually choose one direction in which they want to steer the world", what does that mean, in terms of an approach I can explore?

ETA: I wrote a post that may help explain what I meant here.