The AI-box for hunter-gatherers

Alex Flint

The following is a minor curiosity that occurred to me regarding real-world analogies to the AI-box concept.

Fundamentally, the reason that we fear a randomly-chosen super-intelligent AI is twofold:

It would be smarter than us, so it could outwit us no matter what its goals.
We have no reason to expect its goals to exactly coincide with ours, so we expect its actions to be detrimental to us.

Now, let's say you're playing a game with a wide range of outcomes, over which you have a well-defined utility function. Let's say you can choose a partner from some pool of potential partners, each with a random utility function. Let's say the intelligence of each player is known to you. It would be unwise to choose a player more intelligent than yourself because

They could outwit you.
You have no reason to expect their goals to coincide with yours.

On the other hand, if you pick a less intelligent player then perhaps you could trick them into furthering your own goals, or at least ascertain whether their motives coincide with yours. At the very least you would be able to keep them from subverting your own goal-directed actions.

I conjecture that this explains why people sometimes fear those more intelligent than themselves, and also why people sometimes act dumb. Imagine a hunter-gatherer group considering inviting an outsider to join them. If the outsider's motives are uncertain then the more intelligent the outsider, the less well-advised the group would be to let em in.

In fact, a situation analogous to an AI-box could arise:

Group member 1: We should let this intelligent outsider in, but we should keep tabs on them should they act against us.

Group member 2: But the fact that they're more intelligent than us means that you *can't* expect to keep tabs on them.

Group member 1: But couldn't we set a test of their motives and only allow them in if they proves to have motives coinciding with ours?

Group member 2: No matter what test you set, we can expect them to outwit us -- that's what intelligence means.

The following is a minor curiosity that occurred to me regarding real-world analogies to the AI-box concept.

Fundamentally, the reason that we fear a randomly-chosen super-intelligent AI is twofold:

It would be smarter than us, so it could outwit us no matter what its goals.
We have no reason to expect its goals to exactly coincide with ours, so we expect its actions to be detrimental to us.

They could outwit you.
You have no reason to expect their goals to coincide with yours.

In fact, a situation analogous to an AI-box could arise:

Group member 1: We should let this intelligent outsider in, but we should keep tabs on them should they act against us.

Group member 2: But the fact that they're more intelligent than us means that you *can't* expect to keep tabs on them.

Group member 1: But couldn't we set a test of their motives and only allow them in if they proves to have motives coinciding with ours?

Group member 2: No matter what test you set, we can expect them to outwit us -- that's what intelligence means.

Any smart person who really knows how to control the actions of less intelligent people could potentially make a fortune advising corrections facilities, juvenile halls, and schools with severe chronic discipline problems

I'm in general agreement with your post, but being good at X is quite different from being good at teaching how to do X.

Good point. Anyway, I think we agree that controlling people is not always an easy task, even for people who are very, very smart in other ways.

11

The AI-box for hunter-gatherers

11

11

11

The AI-box for hunter-gatherers

11

11