You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

ahartell comments on Why study the cognitive science of concepts? - Less Wrong Discussion

8 Post author: lukeprog 03 December 2011 11:56PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (4)

You are viewing a single comment's thread. Show more comments above.

Comment author: ahartell 04 December 2011 11:26:58PM 2 points [-]

Most of the discussions at LW about Friendly AI seem to concern defining clear moral rules to program into AIs

I don't think this is true at all. In my understanding the general consensus is that it would not be efficacious to try imposing rules on something that is vastly smarter than you and capable of self-modification. You want it to want to be "friendly" so that it will not (intentionally) change itself away from "friendliness".

It's not a stretch to assume AIs will be capable of some form of self-modification

I agree. Isn't that the basis for AI-based singularitarianism?

and even if it's not their "intention" to do so, copying is never perfect,

I'm pretty sure Eliezer has put a lot of thought into the importance of goal-perserving modification and reproduction.