Posts
Wiki Contributions
Comments
No, introducing the concept of "indexical sample space" does not capture the thirder position, nor language.
And what does it not capture in thirder position, in your opinion?
You do not need to introduce a new type of space, with new definitions and axioms. The notion of credence (as defined in the Sleeping Beauty problem) already uses standard mathematical probability space definitions and axioms.
So thirder think. But they are mistaken, as I show in the previous posts.
Thirder credence fits the No-Coin-Toss problem where Monday and Tuesday don't happen during the same iteration of the experiment and on awakening the person indeed learns that "they are awaken Today", which can be formally expressed as an event .
Not so with Sleeping Beauty, where the participant completely aware that Monday awakening on Tails is followed by Tuesday awakening, therefore, event doesn't happen in 50% cases, so instead of learning that the Beauty is awakened today she can only learn that she is awakened at least once.
In Sleeping Beauty problem being awakened Today isn't a thing you can express via probability space. It's something that can happen twice in the same iteration of the experiment, just like getting a ball in the example from the post. And so we need a new mathematical model to formally talk about this sort of things, therefore weighted probability space.
I suppose you've read all my posts on the topic. What is the crux of our disagreement here?
If this was true, then we could not observe the event „any other coin sequence“ as well since this event is by definition not being tracked.
When you are tracking event A you are automatically tracking its complement.
In fact, in order to detect a correspondence between a coin sequence that we have in mind and the actual sequence, our brain has to compare them to decide if there is a match. I can hardly imagine how this comparison could work without observing the specific actual sequence in the first place. That we classify and perceive a specific sequence as „any other sequence“ can be the result of the comparison, but is not its starting point.
Oh sure, you are of course completely correct here. But this doesn't contradict what I'm saying.
The thing is, we observe a particular outcome and then we see which event(s) it corresponds to. Let's take an example: a series of 3 coin tosses.
So, in the beginning you have sample space which consist of all the elementary outcomes:
And an event space, some sigma-algebra of the sample space, which depends on your precommitments. Normally, it would look something like this:
Because you are intuitively paying attention to whether there all Heads/Tails in a row. So your event space groups individual outcomes in this particular way, separating the event you are tracking and it's complement.
When a particular combination, say is realized in a iteration of the experiment, your mind works like this:
- Outcome is realized
- Therefore every event from the event space which includes is realized.
- Events and are realized.
- This isn't a rare event and so you are not particularly surprised
So, as you see, you do indeed observe an actual sequence, it's just that observing this sequence isn't necessary an event in itself.
Sure, I don‘t deny that. What I am saying is, that your probability model don‘t tell you which probability you have to base on a certain decision
It says which probability you have, based on what you've observed. If you observed that it's Monday - you are supposed to use probability conditionally on the fact that it's Monday, if you didn't observe that it's Monday you can't lawfully use the probability conditionally on the fact that it's Monday. Simple as that.
There is a possible confusion where people may think that they have observed "this specific thing happened" while actually they observed "any thing from some group of things happened", which is the technicolor and rare event cases are about.
Suppose a simple experiment where the experimenter flips a fair coin and you have to guess if Tails or Heads, but you are only rewarded for the correct decision if the coin comes up Tails. Then, of course, you should still entertain unconditional probabilities P(Heads)=P(Tails)=1/2. But this uncertainty is completely irrelevant to your decision.
Here you are confusing probability and utility. The fact that P(Heads)=P(Tails)=1/2 is very much relevant to our decision making! The correct reasoning goes like this:
P(Heads) = 1/2
P(Tails) = 1/2
U(Heads) = 0
U(Tails) = X,
E(Tails) = P(Tails)U(Tails) - P(Heads)U(Heads) = 1/2X - 0
Solving E(Tails) = 0 for X:
X = 0
Which means that you shouldn't bet on Heads at any odds
What is relevant, however, is P(Tails/Tails)=1 and P(Heads/Tails)=0, concluding you should follow the strategy always guessing Tails.
And why did you happen to decide that it's P(Tails|Tails) = 1 and P(Heads|Tails) = 0 instead of
P(Heads|Heads) = 1 and P(Tails|Heads) = 0 which are "relevant" for you decision making?
You seem to just decide the "relevance" of probabilities post hoc, after you've already calculated the correct answer the proper way. I don't think you can formalize this line of thinking, so that you had a way to systematically correctly solve decision theory problems, which you do not yet know the answer to. Otherwise, we wouldn't need utilities as a concept.
Another way to arrive at this strategy is to calculate expected utilities setting U(Heads)=0 as you would propose. But this is not the only reasonable solution. It’s just a different route of reasoning to take into account the experimental condition that your decision counts only if the coin lands Tails.
This is not "another way". This is the right way. It has the proper formalization and actually allows us to arrive to the correct answer even if we do not yet know it.
If the optimal betting sheme requires you to rely on P(Heads/Red or Blue)=1/2 when receiving evidence Blue, then the betting sheme demands you to ignore your total evidence.
You do not "ignore your total evidence" - you are never supposed to do that. It's just that you didn't actually receive the evidence in the first place. You can observe the fact that the room is blue in the experiment only if you put your mind in a state where you distinguish blue in particular. Until then your event space doesn't even include "Blue" only "Blue or Red".
But I suppose it's better to go to the comment section Another Non-Anthropic Paradox for this particular crux.
God is good*
*for a very specific definition of "goodness", which doesn't actually capture the intuition of most people about ethics and is mostly about iteraction of sub-atomic particles.
3First of all, it‘s certainly important to distinguish between a probability model and a strategy. The job of a probability model is simply to suggest the probability of certain events and to describe how probabilities are affected by the realization of other events. A strategy on the other hand is to guide decision making to arrive at certain predefined goals.
Of course. As soon as we are talking about goals and strategies we are not talking about just probabilities anymore. We are also talking about utilities and expected utilities. However, probabilities do not suddenly change because of it. Probabilistic model is the same, there are simply additional considerations as well.
My point is, that the probabilities a model suggests you to have based on the currently available evidence do NOT neccessarily have to match the probabilities that are relevant to your strategy and decisions.
Whether or not your probability model leads to optimal descision making is the test allowing to falsify it. There are no separate "theoretical probabilities" and "decision making probabilities". Only the ones that guide your behaviour can be correct. What's the point of a theory that is not applicable to practice, anyway?
If your model claims that the probability based on your evidence is 1/3 but the optimal decision making happens when you act as if it's 1/2, then your model is wrong and you switch to a model that claims that the probability is 1/2. That's the whole reason why betting arguments are popular.
If Beauty is awake and doesn‘t know if it is the day her bet counts, it is in fact a rational strategy to behave and decide as if her bet counts today.
Questions of what "counts" or "matters" are not the realm of probability. However, the Beauty is free to adjust her utilities based on the specifics of the betting scheme.
All your model suggests are probabilities conditional on the realization of certain events.
The model says that
P(Heads|Red) = 1/3
P(Heads|Blue) = 1/3
but
P(Heads|Red or Blue) = 1/2
Which obviosly translates in a betting scheme: someone who bets on Tails only when the room is Red wins 2/3 of times and someone who bets on Tails only when the room is Blue wins 2/3 of times, while someone who always bet on Tails wins only 1/2 of time.
This leads to a conclusion that observing event "Red" instead of "Red or Blue" is possible only for someone who has been expecting to observe event "Red" in particular. Likewise, observing HTHHTTHT is possible for a person who was expecting this particular sequence of coin tosses, instead of any combination with length 8. See Another Non-Anthropic Paradox: The Unsurprising Rareness of Rare Events
From our state of knowledge about consciousness it's indeed not impossible that modern LLMs are conscious. I wouldn't say it's likely, I definitely wouldn't say that they are as likely to be conscious as uploaded humans. But the point stands. We don't know for sure and we lack proper way to figure it out.
Previously we could've vaguely point towards Turing test, but we are past this stage now. Behavioral analysis of a model at this point is mostly unhelpful. A few tweaks can make the same LLM that previously confidently claimed not to be conscious, to swear that it's conscious and is suffering. So what a current LLM says about the nature of its consciousness gives us about 0 bit of evidence.
This is another reason to stop making bigger models and spend a lot of time figuring out what we have already created. At some point we may create a conscious LLM, won't be able to tell the difference and it would be a moral catastrophe.
You mean, "ban superintelligence"? Because superintelligences are not human-like.
The kind of superintelligence that doesn't possess human-likeness that we want it to possess.
That's the problem with your proposal of "ethics module". Let's suppose that we have system of "ethics module" and "nanotech design module". Nanotech design module outputs 3D-model of supramolecular unholy abomination. What exactly should ethics module do to ensure that this abomination doesn't kill everyone?
Nanotech design module has to be evaluatable by the ethics module. For that it also be made from multiple sequential LLM calls in explicit natural language. Other type of modules should be banned.
indeed. But still… if she wonders out loud “what day is it?” at the very moment she says that, it has an answer.
There is no "but". As long as the Beauty is unable to distinguish between Monday and Tuesday awakenings, as long as the decision process which leads her to say the phrase "what day is it" works the same way, from her perspective there is no one "very moment she says that". On Tails, there are two different moments when she says this, and the answer is different for them. So there is no answer for her
An experimenter who overhears her knows the answer. It seems to me that you “resolve” this tension is that the two of them are technically asking a different question, even though they are using the same words
Yes, you are correct. From the position of the experimenter, who knows which day it is, or who is hired to work only on one random day this is a coherent question with an actual answer. The words we use are the same but mathematical formalism is different.
For an experimenter who knows that it's Monday the probability that today is Monday is simply:
P(Monday|Monday) = 1
For an experimenter who is hired to work only on one random day it is:
P(Monday|Monday xor Tuesday) = 1/2
But still… how surprised should she be if she were to learn that today is Monday? It seems that taking your stance to its conclusion, the answer would be “zero surprise: she knew for sure she would wake up on Monday so no need to be surprised it happened”
And even if she were to learn that the coin landed tails, so she knows that this is just one of a total of two awakenings, she should have zero surprise upon learning the day of the week, since she now knows both awakenings must happen.
Completely correct. Beauty knew that she would be awaken on Monday either way and so she is not surprised. This is a standard thing with non-mutually exclusive events. Consider this:
A coin is tossed and you are put to sleep. On Heads there will be a red ball in your room. On Tails there will be a red and a blue ball in your room. How surprised should you be to find a red ball in your room?
Which seems to violate conservation of expected evidence, except you already said that the there’s no coherent probabilities here for that particular question, so that’s fine too.
The appearance of violation of conservation of expected evidence comes from the belief that awakening on Monday and on Tuesday are mutually exclusive, while they are, in fact sequential.
This makes sense, but I’m not used to it. For instance, I’m used to these questions having the same answer:
- P(today is Monday)?
- P(today is Monday | the sleep lab gets hit by a tornado)
Yet here, the second question is fine (assuming tornadoes are rare enough that we can ignore the chance of two on consecutive days) while the first makes no sense because we can’t even define “today”
It makes sense but it’s very disorienting, like incompleteness theorem level of disorientation or even more
I completely understand. It is counterintuitive because evolution didn't prepare us to deal with situations where an experience is repeated the same while we receive memory loss. As I write in the post:
If I forget what is the current day of the week in my regular life, well, it's only natural to start from a 1/7 prior per day and work from there. I can do it because the causal process that leads to me forgetting such information can be roughly modeled as a low probability occurrence which can happen to me at any day.
It wouldn't be the case, if I was guaranteed to also forget the current day of the week on the next 6 days as well, after I forgot it on the first one. This would be a different causal process, with different properties - causation between forgetting - and it has to be modeled differently. But we do not actually encounter such situations in everyday life, and so our intuition is caught completely flat footed by them.
The whole paradox arises from this issue with our intuition, and just like with incompleteness theorem (thanks for the flattering comparison, btw), what we need to do now is to re-calibrate our intuitions, make it more accustomed to the truth, preserved by the math, instead of trying to fight it.
"here is how to make LLMs more capable but less humanlike, it will be adopted because it makes LLMs more capable".
Thankfully, this is a class of problems that humanity has an experience dealing with. The solution boils down to regulating all the ways to make LLMs less human-like out of existence.
Not just any set. A sample space. And one of its conditions is that its elements are mutually exclusive, so that one and only one happens in any iteration of probability experiment.
That's why I need to define a new mathematical entity indexical sample space, for which I'm explicitly lifting this restriction to formally talk about thirdism.
A minor point is that outcomes and events can both very well be about map not the territory. If elementary event {A} has P(A) = 0, then we can simply not include outcome A into the sample space for simplicity sake.
There is a potential source of confusion in the "credence" category. Either you mean it as a synonym for probability, and then it follows all the properties of probability, including the fact that it can only measure formally defined events from the event space, which have stable truth value during an iteration of probability experiment. Or you mean "intuitive feeling about semantic statements which has some relation to betting", which may or may not be formally definable as probability measure because the statement doesn't have stable truth value.
People tend to implicitly assume that having a vague feeling about a semantic statements has to mean that there is a way to formally define a probability space where this statement is a coherent event, but it doesn't actually has to be true. Sleeping Beauty problem is an example of such situation.
I'm not sure what you mean by "smears whole timelines into one element". We of course should use the appropriate granularity for our outcomes and events. The problem is that we may find ourselves in a situation where we intuitively feel that events has to be even more granular then they formally can.
The fact that something is a semantic statement about the universe doesn't necessary mean that it's well-defined event in a probability space.
No it can't. Semantic statement "Today is Monday" is not a well-defined event in the Sleeping Beauty problem. People can have credence about it in the sense of "vague feeling", but not in the sense of actual probability value.
You can easily observe yourself that there is no formal way to define "Today" in Sleeping Beauty if you actually engage with the mathematical formalism.
Consider No-Coin-Toss or Single-Awakening problems. If Monday means "Monday awakening happens during this iteration of probability experiment" and, likewise, for Tuesday, we can formally define Today as:
Today = Monday xor Tuesday
On every iteration of probability experiment either Monday or Tuesday awakenings happen. So we can say that the participant knows that "she is awakened Today", meaning that she knows to be awakened either on Monday or on Tuesday.
P(Today) = P(Monday xor Tuesday) = 1
We can express credence in being awakened on Monday, conditionally on being awakened "Today" as:
P(Monday|Today) = P(Monday|Monday xor Tuesday) = P(Monday)
This is a clear case where Beauty's uncertainty about which day it is can be expressed via probability theory. Statement "Today is Monday" has stable truth values throughout any iteration of probability experiment
Now consider No-Memory-Loss problem where Sleeping Beauty is completely aware which day it is.
Now statement "Today is Monday" doesn't have a stable truth value throughout the whole experiment. It's actually two different statement: Monday is Monday and Tuesday is Tuesday. The first one is always True, the second one is always False. So Beauty's uncertainty about the question which day it is can't be expressed via probability theory. Thankfully, she doesn't have any uncertainty about the day of the week.
So we can do a trick. We can describe No-Memory-Loss problem as two different non-independent probability experiments in a sequential order. First one is Monday-No-Memory-Loss, where the Beauty is sure that it's is Monday and uncertain about the coin. The second is Tuesday-Tails-No-Memory-Loss where the Beauty is sure that it's Tuesday and the coin is Tails. The second happens only if the coin was Tails in the first.
In Monday-No-Memory-Loss, Today simply means Monday:
Today = Monday
And statement "Today is Monday" is a well defined event with trivial probability measure:
P(Monday|Today) = P(Monday|Monday) = 1
Similarly with Tuesday-Tails-No-Memory-Loss:
Today = Tuesday
P(Monday|Today) = P(Monday|Tuesday) = 0
And now when we consider regular Sleeping Beauty problem the issue should be clear. If we define Today = Monday xor Tuesday, the Beauty can't be sure that this event happens, because on Tails both Monday and Tuesday are realized.
And we can't take advantage of Beauty's lack of uncertainty about the day as before, because now she has no idea what day it is. And so the statement "Today is Monday" is not a well-defined event of the probability space. It doesn't have a coherent truth value during the experiment - it's True xor ( True and False).
We can still talk about events "Monday/Tuesday awakening happens during this iteration of probability experiment".
P(Monday) = 1
P(Tuesday) = 1/2
And we can use them in betting schemes. If the Beauty is proposed to bet on the statement "Today is Monday" she can calculate her optimal odds the standard way:
E(Monday) = P(Monday)U(Monday) - P(Tuesday)U(Tuesday)
Solving E(Monday) = 0:
U(Monday) = U(Tuesday)/2
So 1:2 odds.
And the last question is: What was then this intuitive feeling about the semantic statement "Today is Monday"? For which the answer is - it was about weighted probability that Monday happens in the experiment.