The AI-box for hunter-gatherers

Alex Flint

11 The AI-box for hunter-gatherers

by Alex Flint

2nd Apr 2011

1 min read

11

The following is a minor curiosity that occurred to me regarding real-world analogies to the AI-box concept.

Fundamentally, the reason that we fear a randomly-chosen super-intelligent AI is twofold:

It would be smarter than us, so it could outwit us no matter what its goals.
We have no reason to expect its goals to exactly coincide with ours, so we expect its actions to be detrimental to us.

Now, let's say you're playing a game with a wide range of outcomes, over which you have a well-defined utility function. Let's say you can choose a partner from some pool of potential partners, each with a random utility function. Let's say the intelligence of each player is known to you. It would be unwise to choose a player more intelligent than yourself because

They could outwit you.
You have no reason to expect their goals to coincide with yours.

On the other hand, if you pick a less intelligent player then perhaps you could trick them into furthering your own goals, or at least ascertain whether their motives coincide with yours. At the very least you would be able to keep them from subverting your own goal-directed actions.

I conjecture that this explains why people sometimes fear those more intelligent than themselves, and also why people sometimes act dumb. Imagine a hunter-gatherer group considering inviting an outsider to join them. If the outsider's motives are uncertain then the more intelligent the outsider, the less well-advised the group would be to let em in.

In fact, a situation analogous to an AI-box could arise:

Group member 1: We should let this intelligent outsider in, but we should keep tabs on them should they act against us.

Group member 2: But the fact that they're more intelligent than us means that you *can't* expect to keep tabs on them.

Group member 1: But couldn't we set a test of their motives and only allow them in if they proves to have motives coinciding with ours?

Group member 2: No matter what test you set, we can expect them to outwit us -- that's what intelligence means.

Personal Blog

11

New Comment

Rendering 0/7 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 10:49 PM

Moderation Log