drnickbone comments on New(ish) AI control ideas - LessWrong

24 Post author: Stuart_Armstrong 05 March 2015 05:03PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (14)

You are viewing a single comment's thread. Show more comments above.

Comment author: drnickbone 14 March 2015 09:04:27PM 1 point [-]

Presumably anything caused to exist by the AI (including copies, sub-agents, other AIs) would have to count as part of the power(AI) term? So this stops the AI spawning monsters which simply maximise U.

One problem is that any really valuable things (under U) are also likely to require high power. This could lead to an AI which knows how to cure cancer but won't tell anyone (because that will have a very high impact, hence a big power(AI) term). That situation is not going to be stable; the creators will find it irresistible to hack the U and get it to speak up.

Comment author: Stuart_Armstrong 19 March 2015 01:43:24PM 2 points [-]

I'm looking at ways round that kind of obstacle. I'll be posting them someday if they work.