Nornagest comments on The Friendly AI Game - Less Wrong

38 Post author: bentarm 15 March 2011 04:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (170)

You are viewing a single comment's thread. Show more comments above.

Comment author: Nornagest 10 July 2011 04:59:46AM 2 points [-]

I'm having a hard time coming up with a motivation system that could lead such an AI to developing an acausal decision theory without relying on some goal-like structure that would end up being externally indistinguishable from terms in a utility function. If we stuck a robot with mechanical engineering tools in a room full of scrap parts and gave it an urge to commit novel actions but no utilitarian guidelines for what actions are desirable, I don't think I'd expect it to produce a working nuclear reactor in a reasonable amount of time simply for having nothing better to do.