MugaSofer comments on Morality is Awesome - Less Wrong

86 [deleted] 06 January 2013 03:21PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (437)

You are viewing a single comment's thread. Show more comments above.

Comment author: MugaSofer 06 January 2013 10:33:05PM 0 points [-]

Hell, if I can delete the fact from my memory that my utility function is being deceived, I'd gladly do so - yes, it will bring some momentous negative utility, but it would be a teensy bit greatly offset by the gains, especially stretched over a huge amount of time.

I don't understand this. If your utility function is being deceived, then you don't value the true state of affairs, right? Unless you value "my future self feeling utility" as a terminal value, and this outweighs the value of everything else ...

Comment author: Fadeway 07 January 2013 03:42:49AM 0 points [-]

No, this is more about deleting a tiny discomfort - say, the fact that I know that all of it is an illusion; I attach a big value to my memory and especially disagree with sweeping changes to it, but I'll rely on the pill and thereby the AI to make the decision what shouldn't be deleted because doing so would interfere with the fulfillment of my terminal values and what can be deleted because it brings negative utility that isn't necessary.

Intellectually, I wouldn't care whether I'm the only drugged brain in a world where everyone is flying real spaceships. I probably can't fully deal with the intuition telling me I'm drugged though. It's not highly important - just a passing discomfort when I think about the particular topic (passing and tiny, unless there are starving children). Whether its worth keeping around so I can feel in control and totally not drugged and imprisoned...I guess that's reliant on the circumstances.

Comment author: MugaSofer 07 January 2013 08:55:02PM 0 points [-]

So you're saying that your utility function is fine with the world-as-it-is, but you don't like the sensation of knowing you're in a vat. Fair enough.