You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

accolade comments on I attempted the AI Box Experiment (and lost) - Less Wrong Discussion

47 Post author: Tuxedage 21 January 2013 02:59AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (244)
Tags: ai aibox

Comments (244)

You are viewing a single comment's thread. Show more comments above.

Comment author: JoachimSchipper 21 January 2013 08:40:48AM 16 points [-]

Note that the AI box setting is not one which security-minded people would consider "competent"; once you're convinced that AI is dangerous and persuasive, the minimum safeguard would be to require multiple people to be present when interacting with the box, and to only allow release with the assent of a significant number of people.

It is, after all, much harder to convince a group of mutually-suspicious humans than to convince one lone person.

(This is not a knock on EY's experiment, which does indeed test a level of security that really was proposed by several real-world people; it is a knock on their security systems.)

Comment author: accolade 21 January 2013 01:30:54PM *  4 points [-]

It is, after all, much harder to convince a group of mutually-suspicious humans than to convince one lone person.

That sounds right. Would you have evidence to back up the intuition? (This knowledge would also be useful for marketing and other present life persuasion purposes.)

(

TL;DR: Mo' people - mo' problems?

I can think of effects that could theoretically make it easier to convince a group:

  • For some reason, Boxy might be better skilled at manipulating social/group dynamics than at influencing a lone wolf.
  • More people make the system more complex. Complexity generally increases the likelihood of security holes.
    • Every extra person makes another target and will bring new soft spots to the table, which the AI could pounce on.
  • Supposing that the most competent person available would get the position of the lone Gatekeeper, the average competence would fall when adding more staff.
    • Then the machine could go for an inductive approach - convince the weakest link first, proceed from there with this human ally on her side.
      • Persuaded humans could principally be employed as actuators, e.g. for pressuring, even attacking opposing group members.
        • The lone wolf could be strong against a computer but weak against fellow humans.
        • Surely you will say "But any communication with the terminal will be supervised by everyone!" But that does not strictly make such influence impossible as far as I can tell.
          • Also the superintelligence could get creative, e.g. instill a discussion among the colleagues so that most of them are distracted.

(You could take preemptive measures against these worries, but Boxy might find security holes in every 'firewall' you come up with - an arms race we could win?)

)

Comment author: JoachimSchipper 21 January 2013 02:39:55PM 3 points [-]

My comment was mostly inspired by (known effective) real-world examples. Note that relieving anyone who shows signs of being persuaded is a de-emphasized but vital part of this policy, as is carefully vetting people before trusting them.

Actually implementing a "N people at a time" rule can be done using locks, guards and/or cryptography (note that many such algorithms are provably secure against an adversary with unlimited computing power, "information theoretic security").

Powered by Reddit Powered by Reddit