it would take a lot for the AI to convince me that it has successfully created copies of me which it will torture, much more than just a propensity for telling the truth.
Is the fact that it is fully capable (based on, say, readings of it's processing capabilities, it's ability to know the state of your current mind, etc), and the fact that it has no reason NOT to do what it says (no skin of it's back to torture the subjective "you"s, even if you DON'T let it out, it will do so just on principal).
While it's understandable to say that, today, you aren't in some kind of Matrix, because there is no reason for you to believe so, in the situation of the guard, you DO know that it can do so, and will, even if you call it's "bluff" that the you right now is the original.
While it's understandable to say that, today, you aren't in some kind of Matrix, because there is no reason for you to believe so, in the situation of the guard, you DO know that it can do so, and will, even if you call it's "bluff" that the you right now is the original.