I don't think abramdemski was referring to a "generic" simulation, as if you were running the laws of physics in a computer, and it happens that consciousness arose in the world (like replicators arise in a Game of Life). Or anything like that.
I said "holodeck", if I got the concept well, then in the real world there is a player, and only one, who is running through a simulation, where he is a PC, while all others are non-self-aware NPC. And in this scenario, the player doesn't want it to be clear that he is inside a simulation (in most CRPG there is no in-game evidence that it is just a game, only very few do), but on the other hand, he does want unusual things to happen to him - like winning at the lottery. Even if the sheer luck of the PC is suspicious - like it is in most games and movies.
So the hypothesis "I am in fact a player who is controlling the PC in a game, in which the scenario is that the PC gains the lottery and then can have fun with the money" does make sense. And indeed, if you had a >10^-8 chance of it being true, after winning the lottery, you should have a decent chance of it being true.
But I don't think a 10^-8 prior is really that low for such a scenario. You've a lot of "and" in it, and each "and" does a multiplication of the probabilities...
But I don't think a 10^-8 prior is really that low for such a scenario. You've a lot of "and" in it, and each "and" does a multiplication of the probabilities...
Well, maybe... I suppose it's "difficult to estimate". My intuition is that there will be some "strange possibilities" which make the probability of winning the lottery much higher. But maybe those particular "strange possibilities" have a prior probability significantly lower than 10^-8, since we have to pick them out of the space of possible "strange possibilities"...
I have just finished reading the section on anthropic bias in Nassim Taleb's book, The Black Swan. In general, the book is interesting to compare to the sort of things I read on Less Wrong; its message is largely very similar, except less Bayesian (and therefore less formal-- at times slightly anti-formal, arguing against misleading math).
Two points concerning anthropic weirdness.
First:
If we win the lottery, should we really conclude that we live in a holodeck (or some such)? From real-life anthropic weirdness:
It seems to me that the right way of approaching the question is: before buying the lottery ticket, what belief-forming strategy would we prefer ourselves to have? (Ignore the issue of why we buy the ticket, of course.) Or, slightly different: what advice would you give to other people (for example, if you're writing a book on rationality that might be widely read)?
"Common sense" says that it would be quite silly to start believing some strange theory, just because I win the lottery. However, Bayes says that if we assign greater than 10-8 prior probability to "strange" explanations of getting a winning lottery ticket, then we should prefer them. In fact, we may want to buy a lottery ticket to test those theories! (This would be a very sensible test, which would strongly tend to give the right result.)
However, as a society, we would not want lottery-winners to go crazy. Therefore, we would not want to give the advice "if you win, you should massively update your probabilities".
(This is similar to the idea that we might be persuaded to defect in Prisoner's Dilemma if we are maximizing our personal utility, but if we are giving advice about rationality to other people, we should advise them that cooperating is the optimal strategy. In a somewhat unjustified leap, I suppose we should take the advice we would give to others in such matters. But I suppose that position is already widely accepted here.)
On the other hand, if we were in a position to give advice to people who might really be living in a simulation, it would suddenly be good advice!
Second:
Taleb discusses an interesting example of anthropic bias:
You'll have to read the chapter if you want to know exactly what "argument" is being discussed, but the general point is (hopefully) clear from this passage. If an event was a necessary prerequisite for our existence, then we should not take our survival of that event as evidence for a high probability of survival of such events. If we remember surviving a car crash, we should not take that to increase our estimates for surviving a car crash. (Instead, we should look at other car crashes.)
This conclusion is somewhat troubling (as Taleb admits). It means that the past is fundamentally different from the future! The past will be a relatively "safe" place, where every event has led to our survival. The future is alien and unforgiving. As is said in the story The Hero With A Thousand Chances:
Now, Taleb is saying that we are that hero. Scary, right?
On the other hand, it seems reasonable to be skeptical of a view which presents difficulties generalizing from the past to the future. So. Any opinions?