wedrifid comments on Hedging our Bets: The Case for Pursuing Whole Brain Emulation to Safeguard Humanity's Future - Less Wrong

11 Post author: inklesspen 01 March 2010 02:32AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (244)

Comments (244)

You are viewing a single comment's thread. Show more comments above.

Comment author: wedrifid 03 March 2010 03:46:35AM 2 points [-]

If an AGI reliably understands 'shut down now', it probably also reliably understands 'translate this document into Russian' but that doesn't necessarily mean it can do anything with 'bring about world peace'.

Unfortunately, it can, and that is one of the reasons we have to be careful. I don't want the entire population of the planet to be forcibly sedated.

Comment author: rwallace 03 March 2010 04:11:13AM 0 points [-]

I don't want the entire population of the planet to be forcibly sedated.

Leaving aside other reasons why that scenario is unrealistic, it does indeed illustrate why part of building a system that can reliably figure out what you mean by simple instructions, is making sure that when it's out of its depth, it stops with an error message or request for clarification instead of guessing.

Comment author: wedrifid 03 March 2010 04:35:32AM 1 point [-]

I think the problem is knowing when not to believe humans know what they actually want.

Powered by Reddit Powered by Reddit