Years ago, before coming up with even crazier ideas, Wei Dai invented a concept that I named UDASSA. One way to think of the idea is that the universe actually consists of an infinite number of Universal Turing Machines running all possible programs. Some of these programs "simulate" or even "create" virtual universes with conscious entities in them. We are those entities.

Generally, different programs can produce the same output; and even programs that produce different output can have identical subsets of their output that may include conscious entities. So we live in more than one program's output. There is no meaning to the question of what program our observable universe is actually running. We are present in the outputs of all programs that can produce our experiences, including the Odin one.

Probability enters the picture if we consider that a UTM program of n bits is being run in 1/2^n of the UTMs (because 1/2^n of all infinite bit strings will start with that n bit string). That means that most of our instances are present in the outputs of relatively short programs. The Odin program is much longer (we will assume) than one without him, so the overwhelming majority of our copies are in universes without Odin. Probabilistically, we can bet that it's overwhelmingly likely that Odin does not exist.

You make a lot of interesting points, but how do you apply them to the question at hand: what should you have for dinner, and why?

This is a fascinating topic, and I hope it attracts more commentary. As Bentarm says, it is important and relevant to each of us, yet the topic is fraught with uncertainty, and it is expensive to try to reduce the uncertainty.

I do not believe Taubes. No one book can outweigh the millions of pages of scientific research which have led to the current consensus in the field. Taubes is polemical, argumentative, biased, and one-sided in his presentation. He makes no pretense of offering an objective weighing of the evidence for and against various nutritional hypotheses. He is selling a point of view, plain and simple. No doubt he felt such a forceful approach was necessary given the enormous odds he faces in trying to gain a hearing for his ideas. But the fact remains that the reader must keep in mind that he is only hearing one side of the story.

Weighed against Taubes (and others who have advocated similar positions) we must consider the entire scientific establishment, thousands of researchers who dedicate their lives to the pursuit of knowledge. To believe Taubes, we must believe that these people are basing their entire professional careers on a foundation of falsehoods. Worse, from the lack of impact Taubes' book has had on consensus opinion, we have to imagine that researchers are willfully ignoring the truths that Taubes so convincingly reveals. Nutrition researchers are intentionally lying and covering up the truth in order to protect the false dogma of the field. (Note that this is exactly the same critique of researchers made by global warming skeptics.)

I can't believe that scientists are so dishonest, or that such a cover-up could be executed successfully. I can't imagine how any young, budding nutrition researcher could go to work in a post-Taubes world with a clean conscience, if the book is really as convincing as it claims to be.

My conclusion is that to someone intimately acquainted with the field, Taubes' book is not as persuasive as it appears to the layman.

Now, I will confess that I have some independent reasons to doubt Taubes. But I would prefer not to go into that because IMO the argument I have outlined above is sufficient. Never believe a polemical, one-sided book which has been rejected by the scientific establishment. I offer that as a valid heuristic which has proven correct in the overwhelming majority of cases.

Here is a remarkable variation on that puzzle. A tiny change makes it work out completely differently.

Same setup as before, two private dice rolls. This time the question is, what is the probability that the sum is either 7 or 8? Again they will simultaneously exchange probability estimates until their shared estimate is common knowledge.

I will leave it as a puzzle for now in case someone wants to work it out, but it appears to me that in this case, they will eventually agree on an accurate probability of 0 or 1. And they may go through several rounds of agreement where they nevertheless change their estimates - perhaps related to the phenomenon of "violent agreement" we often see.

Strange how this small change to the conditions gives such different results. But it's a good example of how agreement is inevitable.

I thought of a simple example that illustrates the point. Suppose two people each roll a die privately. Then they are asked, what is the probability that the sum of the dice is 9?

Now if one sees a 1 or 2, he knows the probability is zero. But let's suppose both see 3-6. Then there is exactly one value for the other die that will sum to 9, so the probability is 1/6. Both players exchange this first estimate. Now curiously although they agree, it is not common knowledge that this value of 1/6 is their shared estimate. After hearing 1/6, they know that the other die is one of the four values 3-6. So actually the probability is calculated by each as 1/4, and this is now common knowledge (why?).

And of course this estimate of 1/4 is not what they would come up with if they shared their die values; they would get either 0 or 1.

Let me give an argument in favor of #4, doing what the others do, in the thermometer problem. Now we seem to have them behaving badly. I think in practice many people would in fact look at other thermometers too in making their guesses. So why aren't they doing it? Two possibilities: they're stupid; or they have a good reason to do it. An example good reason: some thermometers don't read properly from a side angle, so although you think you can see and read all of them, you might be wrong. (This could be solved by #3, writing down the average of the cards, but that doesn't work if everyone tries it since everyone is waiting for everyone else to go first.)

Only if we add a stipulation to this problem, that you are usually right when everyone else is wrong, would it be a good idea to buck the crowd. And even then there is the danger that the others may have some private information that supports their seemingly illogical actions.

Actually if Omega literally materialized out of thin air before me, I would be amazed and consider him a very powerful and perhaps supernatural entity, so would probably pay him just to stay on his good side. Depending on how literally we take the "Omega appears" part of this thought experiment, it may not be as absurd as it seems.

Even if Omega just steps out of a taxi or whatever, some people in some circumstances would pay him. The Jim Carrey movie "Yes Man" is supposedly based on a true story of someone who decided to say yes to everything, and had very good results. Omega would only appear to such people.

When I signed up for cryonics, I opted for whole body preservation, largely because of this concern. But I would imagine that even without the body, you could re-learn how to move and coordinate your actions, although it might take some time. And possibly a SAI could figure out what your body must have been like just from your brain, not sure.

Now recently I have contracted a disease which will kill most of my motor neurons. So the body will be of less value and I may change to just the head.

The way motor neurons work is there is an upper motor neuron (UMN) which descends from the motor cortex of the brain down into the spinal cord; and there it synapses onto a lower motor neuron (LMN) which projects from the spinal cord to the muscle. Just 2 steps. However actually the architecture is more complex, the LMNs receive inputs not only from UMNs but from sensory neurons coming from the body, indirectly through interneurons that are located within the spinal cord. This forms a sort of loop which is responsible for simple reflexes, but also for stable standing, positioning etc. Then there other kinds of neurons that descend from the brain into the spinal cord, including from the limbic system, the center of emotion. For some reason your spinal cord needs to know something about your emotional state in order to do its job, very odd.

Like others, I see some ambiguity here. Let me assume that the substrate includes not just the neurons, but the glial and other support cells and structures; and that there needs to be blood or equivalent to supply fuel, energy and other stuff. Then the question is whether this brain as a physical entity can function as the substrate, by itself, for high level mental functions.

I would give this 95%.

That is low for me, a year ago I would probably have said 98 or 99%. But I have been learning more about the nervous system these past few months. The brain's workings seem sufficiently mysterious and counter-intuitive that I wonder if maybe there is something fundamental we are missing. And I don't mean consciousness at all, I just mean the brain's extraordinary speed and robustness.

Another sample problem domain is crossword puzzles:

Don't stop at the first good answer - You can't write in the first word that seems to fit, you need to see if it is going to let you build the other words.

Explore multiple approaches simultaneously - Same idea, you often can think of a few different possible words that could work in a particular area of the puzzle, and you need to keep them all in mind as you work to solve the other words.

Trust your intuitions, but don't waste too much time arguing for them - This one doesn't apply much because usually people don't fight over crossword puzzles.

Go meta - This is a big one, because usually crossword puzzles have a theme, often quite subtle, and if you look carefully you can see how your answers are building as part of a whole. This then gives you another direction to get ideas for possible answers, as things that would go with the theme, rather than just taking the clues literally.

Dissolve the question - Well, I don't know about this, but I suppose if you get frustrated enough you could throw the puzzle into the trash.

Sleep on it - This works well for this kind of puzzle, I find. Coming back to it in the morning you will often make more progress.

Be ready to recognize a good answer when you see it - Once you have enough crossing words in mind you can have good confidence that you are on the right track and go ahead and write those in, even if you don't have good ideas for some of the linked words. You need to recognize that when enough parts come together and your solution makes them fit, that is a strong clue that you are making progress, even if there are still unanswered aspects.