Gunnar_Zarncke comments on Natural selection defeats the orthogonality thesis - Less Wrong

-13 Post author: aberglas 29 September 2014 08:52AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (71)

You are viewing a single comment's thread. Show more comments above.

Comment author: Gunnar_Zarncke 29 September 2014 09:18:11PM 1 point [-]

I don't think that 'any' sufficiently intelligent agent 'clearly' would. It requires at least a solution to the cartesianism problem which is currently unsolved and not every self-optimizing process neccessarily solves this.

Comment author: Kawoomba 29 September 2014 09:32:00PM 2 points [-]

It's just point 3 from Omohundro's The Basic AI drives paper. Didn't think that's controversial around here. I don't think the Cartesian problem is meant to apply to all power levels (since even plain old humans don't drop anvils on their heads, too often), so the 'sufficiently' ought to cover that objection.

Comment author: Gunnar_Zarncke 30 September 2014 09:54:50AM 1 point [-]

even plain old humans don't drop anvils on their heads, too often

But they do and the reason they mostly don't is found in natural selection and not some inevitable convergence of intelligence.