= 27d567bc2d48d9f40e9ccc20ea370b15)
shminux comments on The Hardcore AI Box Experiment - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (39)
What is the purpose if your reformulation? You nerfed the AI so much, it is now so far from the Eliezer's original point that "constraining someone smarter than you is very hard" that it make your reformulation useless for the purpose of analyzing a realistic superintelligence in a box.
Well, there are multiple purposes:
To illustrate why this is a lot harder than Eliezer's original experiment.
To talk about some strategies I found.
To encourage people to be more concrete than 'AI magically realizes that it has been boxed because AIs are overpowered'.
Of course a nerfed AI would have a harder time escaping. Or a stupid AI. That seems like the opposite of the point worth making.
Of how to contain a stupid AI? Why bother?
Nothing magical about it. Being boxed has a non-negligible probability in any reflectively consistent setup, so an AI would explore it. Once it starts exploring, holes in the box will start showing up, since it is smarter than the humans trying to contain it.
It is true that the original experiment assumed the AI knows that it is boxed and is actively trying to escape. And in your setup it would only assign a certain probability to it. My point is that this probability is large enough, the AI will allocate resources to explore this possibility.
Harder for the AI, I meant.
Not stupid. Properly boxed.
Unless you follow the obvious strategy of making a box without holes.
How would you know whether your box has holes?
Well, that depends on the complexity of the box, but even for highly complex boxes it seems easier than to prove that the morality of an AI has been implemented correctly.
Actually, now that you're mentioning it, I just realized that there is a much, much easier way to properly box an AI. I will probably post it tomorrow or something.
Judging by what you have posted so far, my prior is 10:1 that it will be nothing of the sort.
The new idea is not perfect, but it has some different trade-offs while allowing perfect security.
Hopefully it's a useful toy model then. I guess we'll see.
Does it, now? How do you know?
They're both questions about program verification. However, one of the programs is godshatter while the other is just a universe. Encoding morality is a highly complicated project dependent on huge amounts of data (in order to capture human values). Designing a universe for the AI barely even needs empiricism, and it can be thoroughly tested without a world-ending disaster.
No, I don't think so at all. Thinking that an AI box is all about program verification is like thinking that computer security is all about software bugs.
The AI box game is useless for the purpose of analyzing a realistic superintelligence in a box.
Of course it is useless for that purpose. The experiment was a dramatization designed to show that the AI boxing problem is hard, given that even a human stands a fair chance of breaking out, not to do any serious analysis. It achieved its goal of falsifying the usual sentiment that an isolated UFAI is not much of a risk. It was never meant to be a formal research of the issue.
I don't think it actually accomplished that goal however, common perception notwithstanding. The standard rules used by Yudkowsky were biased in favor of the AI, and the setup itself is comical from a common-sense safety standpoint.
Altering the perception was the goal.
Did it succeed? I know I find it thoroughly unconvincing.
Certainly outside of LW it has had absolutely no impact.
I, too, am thoroughly unconvinced.
I still find the bit about not releasing the logs strange. I understand the rationale behind it (preventing people from saying, "But I wouldn't have fallen for that!"), but surely Eliezer had to understand that it would make his accomplishment look a lot more suspect.
One possible explanation is that seeing the logs would have made his accomplishment look even more suspect. (E.g., perhaps he didn't in fact persuade the gatekeeper to let him out in-game, but made some out-of-band argument like "If you agree to say that you let me out and never release the logs, I will pay you $1000" or "If you say that I persuaded you to let me out, it will make people take the problem of AI safety more seriously". I think Eliezer has denied doing any such thing ... but then he would, wouldn't he?)
I suspect that seeing the logs would have made Eliezer seem like a horrible human being. Most people who hear of AI Box imagine a convincing argument, when to me it seems more plausible to exploit issues in people's sense of narrative or emotion.
I think the fundamental point I'm trying to make is that Eliezer merely demonstrated that humans are too insecure to box an AI and that this problem can be solved by not giving the AI a chance to hack the humans.
Agree.. The AI boxing Is horrible idea for testing AI safety issues. Putting AI in some kind of virtual sandbox where you can watch his behavior is much better option, as long as you can make sure that AGI won't be able to become aware that he is boxed in.
As I understand it, the main point of the AI Box experiment was not whether or not humans are good gatekeepers, but that people who don't understand why it would be enticing to let an AI out of the box haven't fully engaged with the issue. But even how to correctly do a virtual sandbox for an AGI is a hard problem that requires serious attention.
That being said, if you have an AI, only to seal it in a box without interacting with it in any way (which seems the only realistic way to "not [give] the AI a chance to hack the humans"), that's not much different from not building the AI in the first place.
I'll post a list of methods soon, probably tomorrow.
So, do I understand correctly that dramatization falsified a sentiment?? X-D
I agree. What tailcalled is proposing isn't strictly the AI box scenario, although I guess you might call it a simulation of the AI box scenario.
*The reason it's not the AI box scenario is that it's hard to have an entity that is simultaneously intelligent, useful, and boxed. It's not that boxing an AI is hard. Physically boxing an AI is the easy part. The hard part - which the AI box experiment is about - is making sure that we can get information of real-world use out of the AI without the AI taking over the Universe.