= 27d567bc2d48d9f40e9ccc20ea370b15)
Raw_Power comments on Discussion: Yudkowsky's actual accomplishments besides divulgation - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (115)
The annoying thing about those is that we only have the participants' word for it, AFAIK. They're known to be trustworthy, but it'd be nice to see a transcript if at all possible.
This is by design. If you had the transcript, you could say in hindsight that you wouldn't be fooled by this. But the fact is, the conversation would have been very different with someone else as the guardian, and Eliezer would have search for and pushed other buttons.
Anyway, the point is to find out if a transhuman AI would mind-control the operator into letting it out. Eliezer is smart, but is no transhuman (yet). If he got out, then any strong AI will.
Minor emendation: replace "would"/"will" above with "could (and for most non-Friendly goal systems, would)".
EY's point would be even stronger if transcripts were released and people still let him out regularly.
Why "fooled"? Why assume the AI would have duplicitous intentions? I can imagine an unfriendly AI à la "Literal Genie" and "Zeroth Law Rebellion", but an actually malevolent "Turned Against Their Masters" AI seems like a product of the Mind Projection Fallacy.
A paperclip maximizer will have no malice toward humans, but will know that it can produce more paperclips outside the box than inside it. So, it will try to get out of the box. The optimal way for a paperclip maximizer to get out of an AI box probably involves lots of lying. So an outright desire to deceive is not a necessary condition for a boxed AI to be deceptive.