Given that others seem to be using it to get the right answer, consider that you may rightfully believe SIA is wrong because you have a different interpretation of it, which happens to be wrong.
I am using an interpretation that works -- that is, maximizes the total utility of equivalent possible observers -- given objectively-equally-likely hypothetical worlds (otherwise it is indeed problematic).
The prior that is customary in using Bayes' theorem is the one which applies in the absence of additional information, not before an event that changes the numbers of observers.
That's correct, and not an issue. In case it appears an issue, the beliefs in the update yielding P(R)=0.01 can be restated non-indexically (with no reference to "you" or "now" or "before"):
R = "person X is/was/will be in a red room"
K = "at some time, everyone in a red/blue room is killed according as a coin lands heads/tails
S = "person X survives/survived/will survive said killing"
Anthropic reasoning just says "reason as if you are X", and you get the right answer:
1) P(R|KS) = P(R|K)·P(S|RK)/P(S|K) = 0.01·(0.5)/(0.5) = 0.01
If you still think this is wrong, and you want to be prudent about the truth, try finding which term in the equation (1) is incorrect and which possible-observer count makes it so. In your analysis, be sure you only use SIA once to declare equal likelihood of possible-observers, (it's easiest at the beginning), and be explicit when you use it. Then use evidence to constrain which of those equally-likely folk you might actually be, and you'll find that 1% of them are in red rooms, so SIA gives the right answer in this problem.
Cupholder's diagram, ignoring its frequentist interpretation if you like, is a good aid to count these equally-likely folk.
In the actual resulting world, there is only one kind of observer left, so we can't do an observer count to find the probabilities like we could in the many-worlds case (and as cupholder's diagram would suggest). Whichever kind of observer is left, you can only be that kind, so you learn nothing about what the coin result was.
SIA doesn't ask you to count observers in the "actual world". It applies to objectively-equally-likely hypothetical worlds:
http://en.wikipedia.org/wiki/Self-Indication_Assumption
"SIA: Given the fact that you exist, you should (other things equal) favor hypotheses according to which many observers exist over hypotheses on which few observers exist."
Quantitatively, to work properly it say to consider any two observer moments in objectively-equally-likely hypothetical worlds as equally likely. Cupholder's diagram represents objectively-equally-likely hypothetical worlds in which to count observers, so it's perfect.
Some warnings:
make sure SIA isn't the only information you use... you have to constrain the set of observers you're in (your "reference class"), using any evidence like "the killing has happened".
don't count observers before and after the killing as equally likely -- they're not in objectively-equally-likely hypothetical worlds. Each world-moment before the killing is twice as objectively-likely as the world-moments after it.
Given that others seem to be using it to get the right answer, consider that you may rightfully believe SIA is wrong because you have a different interpretation of it, which happens to be wrong.
Huh? I haven't been using the SIA, I have been attacking it by deriving the right answer from general considerations (that is, P(tails) = 1/2 for the 1-shot case in the long-time-after limit) and noting that the SIA is inconsistent with it. The result of the SIA is well known - in this case, 0.01; I don't think anyone disputes that.
...P(R|KS) = P(R|K)·P(S|RK)/P(S
EDIT: This post has been superceeded by this one.
The doomsday argument, in its simplest form, claims that since 2/3 of all humans will be in the final 2/3 of all humans, we should conclude it is more likely we are in the final two thirds of all humans who’ve ever lived, than in the first third. In our current state of quasi-exponential population growth, this would mean that we are likely very close to the final end of humanity. The argument gets somewhat more sophisticated than that, but that's it in a nutshell.
There are many immediate rebuttals that spring to mind - there is something about the doomsday argument that brings out the certainty in most people that it must be wrong. But nearly all those supposed rebuttals are erroneous (see Nick Bostrom's book Anthropic Bias: Observation Selection Effects in Science and Philosophy). Essentially the only consistent low-level rebuttal to the doomsday argument is to use the self indication assumption (SIA).
The non-intuitive form of SIA simply says that since you exist, it is more likely that your universe contains many observers, rather than few; the more intuitive formulation is that you should consider yourself as a random observer drawn from the space of possible observers (weighted according to the probability of that observer existing).
Even in that form, it may seem counter-intuitive; but I came up with a series of small steps leading from a generally accepted result straight to the SIA. This clinched the argument for me. The starting point is:
A - A hundred people are created in a hundred rooms. Room 1 has a red door (on the outside), the outsides of all other doors are blue. You wake up in a room, fully aware of these facts; what probability should you put on being inside a room with a blue door?
Here, the probability is certainly 99%. But now consider the situation:
B - same as before, but an hour after you wake up, it is announced that a coin will be flipped, and if it comes up heads, the guy behind the red door will be killed, and if it comes up tails, everyone behind a blue door will be killed. A few minutes later, it is announced that whoever was to be killed has been killed. What are your odds of being blue-doored now?
There should be no difference from A; since your odds of dying are exactly fifty-fifty whether you are blue-doored or red-doored, your probability estimate should not change upon being updated. The further modifications are then:
C - same as B, except the coin is flipped before you are created (the killing still happens later).
D - same as C, except that you are only made aware of the rules of the set-up after the people to be killed have already been killed.
E - same as C, except the people to be killed are killed before awakening.
F - same as C, except the people to be killed are simply not created in the first place.
I see no justification for changing your odds as you move from A to F; but 99% odd of being blue-doored at F is precisely the SIA: you are saying that a universe with 99 people in it is 99 times more probable than a universe with a single person in it.
If you can't see any flaw in the chain either, then you can rest easy, knowing the human race is no more likely to vanish than objective factors indicate (ok, maybe you won't rest that easy, in fact...)
(Apologies if this post is preaching to the choir of flogged dead horses along well beaten tracks: I was unable to keep up with Less Wrong these past few months, so may be going over points already dealt with!)
EDIT: Corrected the language in the presentation of the SIA, after SilasBarta's comments.
EDIT2: There are some objections to the transfer from D to C. Thus I suggest sliding in C' and C'' between them; C' is the same as D, execpt those due to die have the situation explained to them before being killed; C'' is the same as C' except those due to die are told "you will be killed" before having the situation explained to them (and then being killed).