torekp comments on The Need for Human Friendliness - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (28)
I think it's at least as probable that the "simulated" goal set will become the bulk of the real one. That's if one single person starts out with the disguised plan. A conspiracy might be more stable, but also more likely to be revealed before completion.
It's possible, but I think if you're clever about it, you can precommit yourself to reverting to your original preferences once the objective is within your grasp. Every time you accomplish something good or endure something bad, you can tell yourself "All this is for my ultimate goal!" Then when you can actually get either your ultimate goal or your pretend ultimate goal, you'll be able to think to yourself "Remember all those things I did for this? Can't betray them now!" Or some other analogous plan. But I would agree that the trade off here is probably having an easier time pretending vs having greater fidelity to your original goal set. Probaby.