Stuart_Armstrong comments on An overall schema for the friendly AI problems: self-referential convergence criteria - Less Wrong

17 Post author: Stuart_Armstrong 13 July 2015 03:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (110)
Tags: convergence error meta cev fai philosophy motivation

Comments (110)

You are viewing a single comment's thread. Show more comments above.

Comment author: Stuart_Armstrong 15 July 2015 09:53:31AM 0 points [-]

Because FAI's can change themselves very effectively in ways that we can't.

It might be that human brain in computer software would have the same issues.

Comment author: Kaj_Sotala 15 July 2015 01:02:16PM *  2 points [-]

Because FAI's can change themselves very effectively in ways that we can't.

Doesn't mean the FAI couldn't remain genuinely uncertain about some value question, or consider it not worth solving at this time, or run into new value questions due to changed circumstances, etc.

All of those could prevent reflective equilibria, while still being compatible with the ability for extensive self-modification.

Comment author: Stuart_Armstrong 15 July 2015 03:34:23PM 0 points [-]

All of those could prevent reflective equilibria, while still being compatible with the ability for extensive self-modification.

It's possible. They feel very unstable, though.

Powered by Reddit Powered by Reddit