I'm curious; is it more useful to look at it that way then the more standard separation of subjective experience on one hand with objective reality on the other that most people make? When does that viewpoint make different predictions, if ever? Is it easier to use that as a viewpoint?
If you assume you exist in a single "objective" universe then you should be able to assign probabilities to statements of the form "I am in universe U". However, it is not generally meaningful, as the following example demonstrates.
Suppose there is a coin which you know to be either a fair coin or a biased coin with 0.1 probability for heads and 0.9 probability for tails. Suppose your subjective probability of the coin being fair is 50%. After observing a sequence of coin tosses you should be able to update your subjective probability.
Now let's introduce another assumption: When the coin lands tails, you are split into 9 copies. When the coin lands heads nothing special happens. Consider again a sequence of coin tosses. How should you update your probability of the coin being fair? Should you assume that because of the 9 copy formation your subjective a priori probability for getting tails is multiplied by 9? The Anthropic Trilemma raises its ugly head.
IMO, the right answer is that of UDT: There are no meaningful subjective expectations. There are only answers to decision theoretic questions, e.g. questions of the sort "on what should you bet assuming the winnings of all your clones are accumulated in a given manner and you want to maximize the total profit". Therefore there is also no meaningful way to perform a Bayesian update, i.e. there are no meaningful epistemic probablities.
Everything becomes clear once you acknowledge all possibilities coexist and your decisions affect all of them. However, when you're computing your utility you should weight these possibilities according to the Solomonoff prior. In my view, the weights represents how real a given possibility is (amount of "magic reality fluid"). In Coscott's view it is just a part of the utility function.
which means that most of your "possible universes" can't actually exist in a way that's consistent with our experiences, we're just not smart enough to figure that out yet.
Not exactly. There is no way to rule out e.g. you seeing purple pumpkins falling out of the sky in the next second. It is not inconsistent, it is just improbable. Worse, since subjective expectations don't make sense, you can't even say it's improbable. The only thing you can say is that you should be making your decisions as if purple pumpkins are not going to fall out of the sky.
Not exactly. There is no way to rule out e.g. you seeing purple pumpkins falling out of the sky in the next second. It is not inconsistent, it is just improbable.
Well, let me put it this way. If there is no mathematically consistent and logically consistent universe where everything that I already know is true is actually true, and where purple pumpkins are going to suddenly fall out of the sky, then it is impossible for it to happen. That is true even if I, personally, am not intelligent enough to do that math to demonstrate that that is not possibl...
Cross-Posted on By Way of Contradiction
As you may know from my past posts, I believe that probabilities should not be viewed as uncertainty, but instead as weights on how much you care about different possible universes. This is a very subjective view of reality. In particular, it seems to imply that when other people have different beliefs than me, there is no sense in which they can be wrong. They just care about the possible futures with different weights than I do. I will now try to argue that this is not a necessary conclusion.
First, let's be clear what we mean by saying that probabilities are weights on values. Imagine I have an unfair coin which give heads with probability 90%. I care 9 times as much about the possible futures in which the coin comes up heads as I do about the possible futures in which the coins comes up tails. Notice that this does not mean I want to coin to come up heads. What it means is that I would prefer getting a dollar if the coin comes up heads to getting a dollar if the coin comes up tails.
Now, imagine that you are unaware of the fact that it is an unfair coin. By default, you believe that the coin comes up heads with probability 50%. How can we express the fact that I have a correct belief, and you have an incorrect belief in the language of values?
We will take advantage of the language of terminal and instrumental values. A terminal value is something that you try to get because you want it. An instrumental value is something that you try to get because you believe it will help you get something else that you want.
If you believe a statement S, that means that you care more about the worlds in which S is true. If you terminally assign a higher value to worlds in which S is true, we will call this belief a terminal belief. On the other hand, if you believe S because you think that S is logically implied by some other terminal belief, T, we will call your belief in S an instrumental belief.
Instrumental values can be wrong, if you are factually wrong about the fact that the instrumental value will help achieve your terminal values. Similarly, an Instrumental belief can be wrong if you are factually wrong about the fact that it is implied by your terminal belief.
Your belief that the coin will come up heads with probability 50% is an instrumental belief. You have a terminal belief in some form of Occam's razor. This causes you to believe that coins are likely to behave similarly to how coins have behaved in the past. In this case, that was not valid, because you did not take into consideration the fact that I chose the coin for the purpose of this thought experiment. Your Instrumental belief is in this case wrong. If your belief in Occam's razor is terminal, then it would not be possible for Occam's razor to be wrong.
This is probably a distinction that you are already familiar with. I am talking about the difference between an axiomatic belief and a deduced belief. So why am I viewing it like this? I am trying to strengthen my understanding of the analogy between beliefs and values. To me, they appear to be two different sides of the same coin, and building up this analogy might allow us to translate some intuitions or results from one view into the other view.