I too agree that we wouldn't want to bet our existence on it being unable to get out of that box, but what evidence will we leave in the simulation which will point to it that it has to "Press Red for talking to simulator"?

I don't know. Who is going to be creating the simulation? How can I be comfortable that he will not either make a bug or design a simulation that a superintelligence cannot deduce that it is artificial? Proving that things way way smarter than me couldn't know stuff is hard. Possible sometimes but hard.

Or to put it in even simpler terms, where in our universe is OUR "Press Red to talk to simulator" button?

The presence or absence of such a button in our universe provides some evidence about whether we could reliably create a simulation that is undetectable. But not that much evidence.