AI systems up to some high level of intelligence plausibly need to know exactly where they are in space-time in order for deception/"scheming" to make sense as a strategy.
This is because they need to know:
1) what sort of oversight they are subject to 
and
2) what effects their actions will have on the real world

(side note: Acausal trade might break this argument)

There are a number of informal proposals to keep AI systems selectively ignorant of (1) and (2) in order to prevent deception.  Those proposals seem very promising to flesh out; I'm not aware of any rigorous work doing so, however.  Are you?
 

New Answer
New Comment

2 Answers sorted by

Daniel Kokotajlo

Ω452

I know of no rigorous proposals. The general challenge such proposals face is that if you are relying on fooling your AGI about something to keep control over it, and it's constantly and rapidly getting smarter and wiser... that's a recipe for your scheme to fail suddenly and silently (when it stops being fooled), which is a recipe for disaster.

Another type of proposal relies on making it actually true that it might be in a simulation--or to put it more precisely perhaps, making it actually the case that future aligned superintelligences will make simulations so accurate that even a baby superintelligence can't tell the difference. However, two can play at that game; more generally this just becomes a special case of acausal trade stuff which will be wild and confusing and very important once AIs are smart enough to take it seriously.

Not necessarily fooling it, just keeping it ignorant.  I think such schemes can plausibly scale to very high levels of capabilities, perhaps indefinitely, since intelligence doesn't give one the ability to create information from thin air...

2Daniel Kokotajlo
Are you describing something that would fit within my 'Another type of proposal...' category?
4David Scott Krueger (formerly: capybaralet)
No, I was only responding to the the first part.

avturchin

40

I wrote a 'message to any future AI', in which I partly argue that any AI should give significant probability mass to the idea that it is in simulation by more advance AI testing its friendliness. 

1 comment, sorted by Click to highlight new comments since:

People have proposed putting an AI into a simulated environment such that when it thinks it is acting on reality, in actuality it is not.

Is that what you mean by "plausibly need to know exactly where they are in space-time"? If not, what do you mean?