= 27d567bc2d48d9f40e9ccc20ea370b15)
Jiro comments on Leaving LessWrong for a more rational life - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (268)
What's your model of how Einstein in fact arrived at truth, if not via a method that is "an example of how to arrive at truth"? It's obvious the method has to work to some extent, because Einstein couldn't have arrived at a correct view by chance. Is your view that Einstein should have updated less from whatever reasoning process he used to pick out that hypothesis from the space of hypotheses, than from the earliest empirical tests of that hypothesis, contra Einstein's Arrogance?
Or is your view that, while Einstein may technically have gone through a process like that, no one should assume they are in fact Einstein -- i.e., Einstein's capabilities are so rare, or his methods are so unreliable (not literally at the level of chance, but, say, at the level of 1000-to-1 odds of working), that by default you should harshly discount any felt sense that your untested hypothesis is already extremely well-supported?
Or perhaps you should harshly discount it until you have meta-evidence, in the form of a track record of successfully predicting which untested hypotheses will turn out to be correct.
The AI box experiment is a response to the claim 'superintelligences are easy to box, because no level of competence at social engineering would suffice for letting an agent talk its way out of a box'. It functions as an existence proof; if a human level of social competence is already sufficient to talk one's way out of a box with nonzero frequency, then we can't dismiss risk from superhuman levels of social competence.
If you think the claim Eliezer was responding to is silly on priors, or just not relevant (because it would be easy to assess an AI's social competence and/or prevent it from gaining such competence), then you won't be interested in that part of the conversation.
The AI box experiment only serves even as that if you assume that the AI box experiment sufficiently replicates the conditions that would actually be faced by someone with an AI in a box. Also, it only serves as such if it is otherwise a good experiment, but since we are not permitted to see the session transcripts for ourselves, we can't tell if it is a good experiment.
Again, the AI box experiment is a response to the claim "superintelligences are easy to box, because no level of competence at social engineering would suffice for letting an agent talk its way out of a box". If you have some other reason to think that superintelligences are hard to box -- one that depends on a relevant difference between the experiment and a realistic AI scenario -- then feel free to bring that idea up. But this constitutes a change of topic, not an objection to the experiment.
I mean, the experiment's been replicated multiple times. And you already know the reasons the transcripts were left private. I understand assigning a bit less weight to the evidence because you can't examine it in detail, but the hypothesis that there's a conspiracy to fake all of these experiments isn't likely.
Not all relevant differences between an experiment and an actual AI scenario can be accurately characterized as "reason to think that superintelligences are hard to box". For instance, imagine an experiment with no gatekeeper or AI party at all, where the result of the experiment depends on flipping a coin to decide whether the AI gets out. That experiment is very different from a realistic AI scenario, but one need not have a reason to believe that intelligences are hard to box--or even hold any opinion at all on whether intelligences are hard to box--to object to the experimental design.
For the AI box experiment as stated, one of the biggest flaws is that the gatekeeper is required to stay engaged with the AI and can't ignore it. This allows the AI to win by either verbally abusing the gatekeeper to the extent that he doesn't want to stay around any more, or by overwhelming the gatekeeper with lengthy arguments that take time or outside assistance to analyze. These situations would not be a win for an actual AI in a box.
Refusing to release the transcripts causes other problems than just hiding fakery. If the experiment is flawed in some way, for instance, it could hide that--and it would be foolish to demand that everyone name possible flaws one by one and ask you "does this have flaw A?", "does this have flaw B?", etc. in order to determine whether the experiment has any flaws. There are also cases where whether something is a flaw is an opinion that can be argued, and it might be that someone else would consider a flaw something that the experimenter doesn't.
Besides, in a real boxed AI situation, it's likely that gatekeepers will be tested on AI-box experiments and will be given transcripts of experiment sessions to better prepare them for the real AI. An experiment that simulates an AI boxing should likewise have participants be able to read other sessions.