Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Grognor comments on The curse of identity - Less Wrong

125 Post author: Kaj_Sotala 17 November 2011 07:28PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (298)

You are viewing a single comment's thread. Show more comments above.

Comment author: Grognor 17 November 2011 02:59:52PM *  1 point [-]

Would you say that the AI is faulty?

Yes. It might be doing exactly what it was designed to do, but its designer was clearly stupid or cruel and had different goals than I'd prefer the AI to have.

Extrapolate this to humans. Humans wouldn't care so much about status if it weren't for flaws like scope insensitivity, self-serving bias, etc., as well as simply poor design "goals".

Comment author: [deleted] 17 November 2011 03:38:32PM -1 points [-]

Yes. It might be doing exactly what it was designed to do, but its designer was clearly stupid or cruel and had different goals than I'd prefer the AI to have.

Where are you getting your goals from? What are you, except your design? You are what Azathoth build. There is no ideal you that you should've become, but which Azathoth failed to make.

Comment author: Grognor 17 November 2011 03:46:36PM 1 point [-]

Azathoth designed me with conflicting goals. Subconsciously, I value status, but if I were to take a pill that made me care entirely about making the world better and nothing else, I would. Just because "evolution" built that into me doesn't make it bad, but it definitely did not give me a coherent volition. I have determined for my self which parts of humanity's design are counterproductive, based on the thousand shards of desire.

Comment author: fubarobfusco 18 November 2011 05:11:43AM 2 points [-]

if I were to take a pill that made me care entirely about making the world better and nothing else, I would.

Would you sign up to be tortured so that others don't suffer dust specks?

("If we are here to make others happy, what are the others here for?")

Comment author: [deleted] 19 November 2011 02:44:21AM 1 point [-]

A better analogy would be asking about a pill that caused pain asymbolia.

Comment author: Grognor 18 November 2011 10:45:09AM 1 point [-]

Yes.