Matt_Simpson comments on Open Thread, May 1-15, 2012 - Less Wrong

7 Post author: OpenThreadGuy 01 May 2012 04:14AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (264)

You are viewing a single comment's thread. Show more comments above.

Comment author: Matt_Simpson 02 May 2012 07:39:38PM 0 points [-]

The conclusion doesn't follow. Ripping apart your body to use the atoms to construct something terminally useful is also instrumentally useful.

Comment author: gRR 02 May 2012 07:44:51PM 0 points [-]

Only if there's general lack of atoms around. When atoms are in abundance, it's more instrumentally useful to ask me for help constructing whatever you find terminally useful.

Comment author: Matt_Simpson 02 May 2012 07:55:27PM 0 points [-]

Right, but your conclusion still doesn't follow - my example was just to show the flaw in your logic. Generally, you have to consider the trade-offs between cooperating and doing anything else instead.

Comment author: gRR 02 May 2012 08:09:49PM -1 points [-]

Well, of course. But which my conclusion you mean that doesn't follow?

Comment author: Matt_Simpson 02 May 2012 08:44:34PM 0 points [-]

Once general intelligence arises as a result of an evolutionary process, it will apply itself to optimizing the (unnecessary) pain [of others] away.

Comment author: gRR 02 May 2012 09:02:42PM 0 points [-]

But "[of others]" part is unnecessary. If every intelligent agent optimizes away their own unnecessary pain, it is sufficient for the conclusion. Unless, of course, there exists a significant number of intelligent agents that have pain of others as a terminal goal, or there's a serious lack of atoms for all agents to achieve their otherwise non-contradicting goals.

Comment author: Matt_Simpson 02 May 2012 09:17:15PM 1 point [-]

If every intelligent agent optimizes away their own unnecessary pain, it is sufficient for the conclusion.

This is highly dependent on the strategic structure of the situation.