If the AI can't tell that it is in a simulation given that it is in a simulation, why would it be able to tell that it is in the real world given that it is in the real world? If such a proposition is effective in a simulation, humans would likely just make the same threat/test in the real world. If the AI forgoes this particular opportunity at world domination because it might be in a simulation, what would make it reason differently in the real world?
ETA: and if the AI can tell if its in a simulation vs the real world then its not really a test at all.
If the AI forgoes this particular opportunity at world domination because it might be in a simulation, what would make it reason differently in the real world?
Hopefully nothing. An AI that plays nice out of the fear of God is still an AI that plays nice.
Top ten tips for an AI bent on destroying humanity: