= 27d567bc2d48d9f40e9ccc20ea370b15)
Kawoomba comments on Natural selection defeats the orthogonality thesis - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (71)
It's kind of interesting that humans generally don't guard themselves against value drift. Even though any sufficiently intelligent agent clearly would. One of those fundamental divides higher up on the intelligence scale than us, divides that seem binary rather than linear in nature. I wonder if there are any more of those. Apart from (a lack of) susceptibility to the usual biases.
I don't think that 'any' sufficiently intelligent agent 'clearly' would. It requires at least a solution to the cartesianism problem which is currently unsolved and not every self-optimizing process neccessarily solves this.
It's just point 3 from Omohundro's The Basic AI drives paper. Didn't think that's controversial around here. I don't think the Cartesian problem is meant to apply to all power levels (since even plain old humans don't drop anvils on their heads, too often), so the 'sufficiently' ought to cover that objection.
But they do and the reason they mostly don't is found in natural selection and not some inevitable convergence of intelligence.