ArisKatsaris comments on SMBC comic: poorly programmed average-utility-maximizing AI - Less Wrong

9 Post author: Jonathan_Graehl 06 April 2012 07:18AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (113)

You are viewing a single comment's thread. Show more comments above.

Comment author: ArisKatsaris 06 April 2012 02:41:28PM *  1 point [-]

Imagine that there is one tortured for 50 years and then free of any dust speck for the next 3^^^3 years.

If I understand the dilemma, in your most recent phrasing, it's this: A person who lives 3^^^3 years either:
a) has to suffer a dustspeck per year
b) has to suffer 50 years of torture at some point in that time, then I assume gets the memory of that torture deleted from his mind and his mind's state restored to what it was before the torture (so that he doesn't suffer further disutility from that memory or the broken mind-state, he only has to suffer the torture itself), He lives the remaining 3^^^3 years dustspeck-free.

If we don't know what his own preferences are, and have no way of asking him, what should we choose on his behalf?

But what does this have to do with Felix?

Comment author: Thomas 06 April 2012 02:52:00PM -1 points [-]

If we don't know what his own preferences are, and have no way of asking him, what should we choose on his behalf?

It is argued in the said sequence, how much better is to have 1 tortured for 50 years, than 3^^^3 people having slight discomfort.

Which preferences are in question now?

Comment author: ArisKatsaris 06 April 2012 03:08:01PM *  2 points [-]

Can we have one dilemma at a time, please, Thomas? You said something about 3^^^3 years -- therefore you're not talking about the dilemma as stated in the original sequence, as that dilemma doesn't say anything about 3^^^3 years.

Which preferences are in question now?

The preferences relating to the original dilemma, are the preferences of the person who presumably prefers not to get tortured, vs the preferences of 3^^^3 people who presumably prefer not to get a dust speck in the eye.