You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Dmytry comments on Yet another safe oracle AI proposal - Less Wrong Discussion

2 Post author: jacobt 26 February 2012 11:45PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (33)

You are viewing a single comment's thread.

Comment author: Dmytry 27 February 2012 02:49:55AM *  1 point [-]

Currently, the optimizer must completely derive a new optimizer from the specification (program 3), without looking at the current best optimizer (its own source code).

....

This improvement should make program 2 even better at bettering itself, in addition to doing better on other optimization problems.

Then call this 'optimizer' a 'superhumanly strong AI', to avoid confusing yourself.