= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on The Importance of Self-Doubt - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (726)
There's some part of my brain that just processes "the Internet" as a single person and wants to scream "But I told you this a thousand times already!"
http://yudkowsky.net/singularity/aibox
Surely it's possible to imagine a successfully boxed AI.
I could imagine successfully beating Rybka at chess too. But it would be foolish of me to take any actions that considered it as a serious possibility. If motivated humans cannot be counted on to box an Eliezer then expecting a motivated, overconfident and prestige seeking AI creator to successfully box his AI creation is reckless in the extreme.
What Eliezer seemed to be objecting to was someone proposing a successfully boxed AI as an example of why "able to destroy humanity" can't be a part of the definition of "AI" (or more charitably, "artificial superintelligence"). For boxed AI to be such an example (as opposed to a good idea to actually strive toward), it only has to be not knowably impossible.
I see your point there. But I think this discussion sort of went in an irrelevant direction, albeit probably my fault for not being clear enough. When I put "powerful enough to destroy humanity" in that criterion, I mainly meant "powerful" as in "really powerful optimization process", mathematical optimization power, not "power" as in direct influence over the world. We're inferring that the former will usually lead fairly easily to the latter, but they are not identical. So "powerful enough to destroy humanity" would mean something like "powerful enough to figure out a good subjunctive plan to do so given enough information about the world, even if it has no output streams and is kept in an airtight safe at the bottom of the ocean".
Reading back further into the context I see your point. Imagining such an AI is sufficient and Eliezer does seem to be confusing a priori with obvious. I expect that he just completed a pattern based off "AI box" and so didn't really understand the point that was being made - he should have replied with a "Yes - But". (I, of course, made a similar mistake in as much as I wasn't immediately prompted to click back up the tree beyond Eliezer's comment.)
Eliezer, while you're defending yourself from charges of self-aggrandizement, it troubles me a little bit that AI Box page states that your record is 2 for 2, and not 3 for 5.
Obviously I'm not trying to keep it a secret. I just haven't gotten around to editing.
I'm sure that's the case, I'm just saying it looks bad. Presumably you'd like to be Caesar's wife?
Move it up your to-do list, it's been incorrect for a time that's long enough to look suspicious to others. Just add a footnote if you don't have time to give all the details.
Thx for the link. If I already had already known the link, I would have asked for it by name. :)
Eliezer, you have written a lot. Some people have read only some of it. Some people have read much of it, but forgotten some. Keep your cool. This situation really ought not to be frustrating to you.
Oh, I know it's not your fault, but seriously, have "the Internet" ask you the same question 153 times in a row and see if you don't get slightly frustrated with "the Internet".
Yeah, after reading your "some part of my brain" thing a second time, I realized I had misinterpreted. Though I will point out that my question was not directed to you. You should learn to delegate the task of becoming frustrated with the Internet.
I read the article (though not yet any of the transcripts). Very interesting. I hope that some tests using a gatekeeper committee are tried someday.
Computer programmers do not normally test their programs by getting a committee of humans to hold the program down - the restraints themselves are mostly technological. We will be able to have the assistance of technological gatekeepers too - if necessary.
Today's prisons have pretty configurable security levels. The real issue will probably be how much people want to pay for such security. If an agent does escape, will it cause lots of damage? Can we simply disable it before it has a chance to do anything undesirable? Will it simply be crushed by the numerous powerful agents that have already been tested?