You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

timtyler comments on Value evolution - Less Wrong Discussion

14 Post author: PhilGoetz 08 December 2011 11:47PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (111)

You are viewing a single comment's thread. Show more comments above.

Comment author: timtyler 09 December 2011 03:43:44PM *  2 points [-]

How does a self-improving system improve itself, without discovering contradictions or gaps in its values?

By getting a faster brain, more memory, more stored resources and a better world model, perhaps.

Values don't have to have "contradictions" or "gaps" in them. Say you value printing out big prime numbers. Where are the contradictions or gaps going to come from?

Does value freeze require knowledge freeze?

Usually values and knowledge are considered to be orthogoonal - so "no".