I know that's your idea, I'm saying it's stupid. If I torture you every night and wipe your memory before morning, are you just indifferent to that? I could add this to the torture: "I asked your daylight self after the mindwipe if it would be wrong to do what I'm doing to you, and he said no, because by black-box reasoning torturing you now doesn't matter, so long as I erase the effects by morning."
ETA: Maybe it's harsh to call it stupid when your original scenario wasn't about deliberately ignoring torture inside the black box. It was just an innocent exercise in being copied and then one of you deleted.
But you cannot presume that the person who anticipates surviving with certainty is correct, just because a copy of them certainly survives to get the bigger payoff. Your argument is: hey cryonics skeptic, here we see someone with a decision procedure which identifies the original with its copies, and it gets the bigger payoff; so judged by the criterion of results obtained ("winning") this is the superior attitude, therefore the more rational attitude, and so your objection to cryonics is irrational.
However, this argument begs the question of whether the copy is the same person as the original. A decision procedure would normally be regarded as defective if it favors an outcome because of mistaken identity - because person X gets the big payoff, and it incorrectly identifies X with the intended beneficiary of the decision making. And here I might instead reason as follows: that poor fool who volunteers for iterated russian roulette, the setup has fooled him into thinking that he gets to experience the payoff, just because a copy of him does.
As I recently wrote here, there is a "local self", the "current instance" of you, and then there may be a larger "extended self" made of multiple instances with which your current instance identifies. In effect, you are asking people to adopt a particular expansive identity theory - you want them to regard their copies as themselves - because it means bigger payoffs for them in your thought-experiment. But the argument is circular. For someone with a narrow identity theory ("I am only my current instance"), to run the gauntlet of iterated russian roulette really is to make a mistake.
The scenario where we torture you and then mindwipe you is not an outright rebuttal of an expansive attitude to one's own personal identity, but it does show that the black-box argument is bogus.
And your edit leaves you with an interesting conundrum.
It can put you in a situation where you see people around yourself adopting one of two strategies, and the people who adopt one strategy consistently win, and the people who adopt another strategy consistently lose, but you still refuse to adopt the winning strategy because you think the people who win are .. wrong.
I'm not sure if you can call that a win.
This is a thought exercise I came up with on IRC to help with the iffiness of "freezing yourself for a thousand years" with regards to continuity of self.
Let's say we live in a post-singularity world as uploads and are pretty bored and always up for terrible entertainment (our god is FAI but has a scary sense of humor ..). So some crazy person creates a very peculiar black cube in our shared reality. You walk into it, a fork of you is created and you duke it out via russian roulette. The winner walks out the other side.
Before entering, should you accurately anticipate dying with 50% probability?
I argued that you should anticipate surviving with 100% probability, since the single you that walked out of the box would turn out to be correct in his prediction. Surprisingly, someone disagreed.
So I extended the scenario by another black box with two doors, but this one is just a tunnel. In this case, everybody can agree that you should anticipate a 100% probability of surviving it unscathed. But if we delete our memory of what just happened when exiting the black boxes, and the boxes themselves, then the resulting universes would be indistinguishable!
One easy way to demonstrate this is to chain ten boxes and put a thousand dollars at the end. The person that anticipates dying with 50% probability (so over all the boxes, 1/1024 chance of surviving) would stay well outside. The person that anticipates surviving just walks through and comes away $1000 richer. "But at least my anticipation was correct", in this scenario, reminds me somewhat of the cries of "but at least my reasoning was correct" on the part of two-boxers.
What I'm wondering is: is there a general rule underlying this about the follies of allowing causally-indistinguishable-in-retrospect effects to differently affect our anticipation? Can somebody formalize this?