You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

lavalamp comments on Yet another safe oracle AI proposal - Less Wrong Discussion

2 Post author: jacobt 26 February 2012 11:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (33)
Tags: oracle python safety agi

Comments (33)

You are viewing a single comment's thread.

Comment author: lavalamp 27 February 2012 05:05:00AM 0 points [-]

In other words, optimizer_score should not be seen as a utility function because the system only tries to improve it using greedy methods, not long-term planning.

It sounds like that would doom it to wallow forever in local maxima.

Comment author: jacobt 27 February 2012 05:32:52AM 0 points [-]

I mean greedy on the level of "do you best to find a good solution to this problem", not on the level of "use a greedy algorithm to find a solution to this problem". It doesn't do multi-run planning such as "give an answer that causes problems in the world so the human operators will let me out", since that is not a better answer.

Powered by Reddit Powered by Reddit