But ... there is a difference between "something that actually is bad, and something I just think is bad." If Omega told me sacrificing a child was the best option by my preferences, I would do so (or at least accept that I should; I would probably experience a lot of akrasia.) Wouldn't you?
There absolutely is a difference, yes.
But if the answer to "how do I tell the difference?" is that I ask the entity who is making the request in the first place, we've now achieved full epistemic closure.
That is: if I don't know whether Omega tells the truth or not, and I don't know whether Omega has my best interests in mind or not, and Omega tells me to sacrifice a child, I probably wouldn't sacrifice the child. Would you?
More generally, there is a big difference between "what ought I do, if X is the case?" and "what decision w...
Happy New Year! Here's the latest and greatest installment of rationality quotes. Remember: