Do you really have that preference? For example, if every but one of trillions of humans was being tortured and had dust specks, would you feel like trading the torture-free human's freedom from torture for the removal of specks from the tortured. If that were so, then you just are showing a fairly usual preference (inequality is bad!) which is probably fine as an approximation of stuff you could formalize consequentially. But that's just an example. Often there's some context in which your moral intuition is reversed, which is a useful probe. (usual caveat: haven't read the sequences) Topic for discussion: Less Wrongians are frequentists to a greater extent than most folk who are intuitively Bayesian. The phrase "I must update on" is half code for (p<0.05) and half signalling, since presumably you're "updating" a lot, just like regular humanssssssssssssssssssssssssssssssssss.

If I recall correctly Alicorn made a reference to reversing the utilities in this argument... would you think it better for someone to give up a life of the purest and truest happiness, if in exchange they created all of the 10 second or less cat videos that will ever be on youtube throughout all of history and the future? My intuitions here say yes; it can be worth sacrificing your life (i.e. torturing yourself working at a startup) to create a public good which will do a small amount for a lot of people (i.e. making standard immunization injections also give people immunity to dust specks in their eyes)