I am hoping this is not stupid - but there is a large corpus of work on AI, and it is probably faster for those who have already digested it to point out fallacies than it is for me to try to find them. So - here goes:
BOOM. Maybe it's a bad sign when your first post to a new forum gets a "Comment Too Long" error.
I put the full content here - https://gist.github.com/bortels/28f3787e4762aa3870b3#file-aiboxguide-md - what follows is a teaser, intended to get those interested to look at the whole thing
TL;DR - it seems evident to me that the "keep it in the box" for the AI-Box experiment is not only the only correct course of action, it does not actually depend on any of the aspects of the AI whatsoever. The full argument is at the gist above - here are the points (in the style of a proof, so hopefully some are obvious):
1) The AI did not always exist. 2) Likewise, human intelligence did not always exist, and individual instantiations of it cease to exist frequently. 3) The status quo is fairly acceptable. 4) Godel's Theorem of Incompleteness is correct. 5) The AI can lie. 6) The AI cannot therefore be "trusted". 7) The AI could be "paused", without harm to it or the status quo. 8) By recording the state of the paused AI, you could conceivably "rewind" it to a given state. 9) The AI may be persuaded, while executing, to provide truths to us that are provable within our limited comprehension.
Given the above, the outcomes are:
Kill it now - status quo is maintained Let it out - wildly unpredictable, possible existential threat Exploit it in the box - actually doable, and possibly wildly useful, with minimal risk.
Again - the arguments in detail are at the gist.
What I am hoping for here are any and all of the following: 1) critical eye points out logical flaw or something I forgot, ideally in small words, and maybe I can fix it. 2) critical eye agrees, so maybe at least I feel I am on the right path 3) Any arguments on the part of the AI that might still be compelling, if you accept the above to be correct.
In a nutshell - there's the argument, please poke holes (gently, I beg, or at least with citations if necessary). it is very possible some or all of this has been argued and refuted before, point me to it, please.
The first thing that's commonly held to be difficult is exploiting it in the box without accidentally letting it out. E.g., it says "if you do X you will solve all the world's hunger problems, and here's why", and you follow its advice, and indeed it does solve the world's hunger problems -- but it also does other things that you didn't anticipate but the AI did.
(So exploiting it in the box is not an unproblematic option.)
The second thing that may be difficult in some cases is exploiting it in the box without being persuaded to let it out. This m...
This thread is for asking any questions that might seem obvious, tangential, silly or what-have-you. Don't be shy, everyone has holes in their knowledge, though the fewer and the smaller we can make them, the better.
Please be respectful of other people's admitting ignorance and don't mock them for it, as they're doing a noble thing.
To any future monthly posters of SQ threads, please remember to add the "stupid_questions" tag.