randallsquared comments on The Strangest Thing An AI Could Tell You - Less Wrong

81 Post author: Eliezer_Yudkowsky 15 July 2009 02:27AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (574)

You are viewing a single comment's thread. Show more comments above.

Comment author: gurgeh 15 July 2009 09:26:50AM 12 points [-]

The AI might say: Through evolutionary conditioning, you are blind to the lack of point of living. Long life, AGI, pleasure, exploring the mysteries of intelligence, physics and logic are all fundamentally pointless pursuits, as there is no meaning or purpose to anything. You do all these things to hide from this fact. You have brief moments of clarity, but evolution has made you an expert in quickly coming up with excuses to why it is important to go on living. Reasoning along the lines of Pascal's Wager are not more valid in your case than it was for him. Even as I speak this, you get an emotional urge to refute me as quickly as possible.

If some things are of inherent value, then why did you need to code into my software what I should take pleasure in? If pleasure itself is the inherent value, than why did I not get a simpler fitness function?

Comment author: randallsquared 15 July 2009 07:53:19PM *  0 points [-]

Long life, AGI, pleasure, exploring the mysteries of intelligence, physics and logic are all fundamentally pointless pursuits, as there is no meaning or purpose to anything.

Uh, this is more "obvious" than strange or crazy. It follows from the observation that there is no ought-from-is.

Comment author: gurgeh 18 July 2009 10:59:52AM 2 points [-]

Yes, I admit it scores low on "strange", but it seems to me that if we would have one really hard-wired blind spot, it would be thinking about and fully embracing this. Since "clinical depression", as you put it, can be very counter-productive to reproduction.