= 27d567bc2d48d9f40e9ccc20ea370b15)
Nick_Tarleton comments on How Not to be Stupid: Know What You Want, What You Really Really Want - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (39)
The simplest way to show the confusion here is just to ask: if those are your only two options (or both obviously dominate all other options – say, Omega will blow up the world if you don't pick one, though it might be best to avoid invoking Omega when you can avoid it), how do you choose?
What is invalid with answering? "By performing further computation and evidence gathering."
And if Omega doesn't give that option, then that significantly changes the state of the world, and hence your priority function - including the priority of assigning the relative priority between A and B.
As I said on the top level post, you can't treat this priority assignment as non-self-referential.
Edited to add: You should not call people confused because they don't want to cache thoughts for which they do not as of yet know the answer.
Yes, the question only applies to final, stable preferences – but AFAIK not everyone agrees that final, stable preferences should be totally ordered.
How do you conclude that a preference is final and stable?
That seems an extremely strong statement to be making about the inner workings of your own mind.
I don't believe any of my own preferences are final and stable. The intent is to characterize the structure that (I believe) an idealized agent would have / that the output of my morality has / that I aim at (without necessarily ever reaching).