Nornagest comments on Everybody's talking about machine ethics - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (16)
Superintelligence doesn't, but recursive self-modification has been a feature of AI research since the Seventies. As MIRI predicts, value stability proved to be a problem.
(Eurisko's authors solved it -- allegedly, there isn't much open data -- by walling off the agent's utility function from modification. This would be much harder to do to a superintelligent agent.)