You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

ike comments on Summoning the Least Powerful Genie - Less Wrong Discussion

-1 Post author: Houshalter 16 September 2015 05:10AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (48)

You are viewing a single comment's thread. Show more comments above.

Comment author: ike 16 September 2015 08:53:28PM *  0 points [-]

Do you have a valid argument that nonagentive programmes would be dangerous? Because saying "it would agentively do X" isn't a valid argument. Pointing out the hidden pitfalls of such programmes is something MIRI could usefully do. An unargued belief that everything is dangerous is not useful.

I'm claiming that "nonagent" is not descriptive enough to actually build one. You replied that we already have non agents, and I replied that we already have safe computer programs. Just like we can't extrapolate from our safe programs that any AI will be safe, we can't extrapolate from our safe nonagents that any non-agent will be safe.

Well: how likely is an AI designed to be nonagentive as a safety feature to have that particular failure mode?

I still have little idea what you mean by nonagent. It's a black box, that may have some recognizable features from the outside, but doesn't tell you how to build it.

Comment author: TheAncientGeek 16 September 2015 09:19:49PM *  1 point [-]

I replied that we can already build nonagents.

It remains the case that if you think they could be dangerous, you need to explain how.

I still have little idea what you mean by nonagent. It's a black box, that may have some recognizable features from the outside, but doesn't tell you how to build it.

Again, we already know how to build them, in that we have them.

Worse than that. MIRI can't actually build anything they propose. It's just that some MIRI people have a reflex habit of complaining that anything outside of MIRI land is too vague.