Comment author: Manfred 07 September 2015 07:57:09PM *  0 points [-]

What makes something an "outcome" in Savage's theorem is simply that it follows a certain set of rules and relationships - the interpretation into the real world is left to the reader.

It's totally possible to regard the state of the entire universe as the "outcome" - in that case, the things that corresponds to the "actions" (the thing that the agent chooses between to get different "outcomes") are actually the strategies that the agent could follow. And the thing that the agent always acts as if it has probabilities over are the "events," which are the things outside the agent's control that determine the mapping from "actions" to "outcomes," and given this interpretation the day does not fulfill such a role - only the coin.

So in that sense, you're totally right. But this interpretation isn't unique.

It's also a valid interpretation to have the "outcome" be whether Sleeping Beauty wins, loses, or doesn't take an individual bet about what day it is (there is a preference ordering over these things), the "action" being accepting or rejecting the bet, and the "event" being which day it is (the outcome is a function of the chosen action and the event).

Here's the point: for all valid interpretations, a consistent Sleeping Beauty will act as if she has probabilities over the events. That's what makes Savage's theorem a theorem. What day it is is an event in a valid interpretation, therefore Sleeping Beauty acts as if it has a probability.

Side note: It is possible to make what day it is a non-"event," at least in the Savage sense. You just have to force the "outcomes" to be the outcome of a strategy. Suppose Sleeping Beauty instead just had to choose A or B on each day, and only gets a reward if her choices are AB or BA, but not AA or BB (or any case where the reward tensor is not a tensor sum of rewards for individual days). To play this game well, Savage's theorem does not say you have to act like you assign a probability to what day it is. The canonical example of this problem in anthropics is the absent-minded driver problem - compared to Sleeping Beauty, it is strictly trickier to talk about whether the absent-minded driver should have a probability that they're at the first intersection - argument in favor have to either resort to Cox's theorem (which I find more confusing), or engage in contortions about games that counterfactually could be constructed.

Comment author: Squark 11 September 2015 06:41:30AM *  0 points [-]

It's also a valid interpretation to have the "outcome" be whether Sleeping Beauty wins, loses, or doesn't take an individual bet about what day it is (there is a preference ordering over these things), the "action" being accepting or rejecting the bet, and the "event" being which day it is (the outcome is a function of the chosen action and the event).

In Savage's theorem acts are arbitrary functions from the set of states to the set of consequences. Therefore to apply Savage's theorem in this context you have to consider blatantly inconsistent counterfactuals in which the sleeping beauty makes difference choices in computationally equivalent situations. If you have an extension of the utility function to these counterfactuals and it happens to satisfy the conditions of Savage's theorem then you can assign probabilities. This extension is not unique. Moreover, in some anthropic scenarios in doesn't exist (as you noted yourself).

...argument in favor have to either resort to Cox's theorem (which I find more confusing), or engage in contortions about games that counterfactually could be constructed.

Cox's theorem only says that any reasonable measure of uncertainty can be transformed into a probability assignment. Here there is no such measure of uncertainty. Different counterfactual games lead to different probability assignments.

In response to comment by Squark on MIRI's Approach
Comment author: YVLIAZ 07 September 2015 06:39:30AM 0 points [-]

There is difference between "having an idea" and "solid theoretical foundations". Chemists before quantum mechanics had a lots of ideas. But they didn't have a solid theoretical foundation.

That's a bad example. You are essentially asking researchers to predict what they will discover 50 years down the road. A more appropriate example is a person thinking he has medical expertise after reading bodybuilding and nutrition blogs on the internet, vs a person who has gone through medical school and is an MD.

In response to comment by YVLIAZ on MIRI's Approach
Comment author: Squark 07 September 2015 06:14:33PM 0 points [-]

I'm not asking researchers to predict what they will discover. There are different mindsets of research. One mindset is looking for heuristics that maximize short term progress on problems of direct practical relevance. Another mindset is looking for a rigorously defined overarching theory. MIRI is using the latter mindset while most other AI researchers are much closer to the former mindset.

Comment author: Manfred 06 September 2015 06:01:49PM *  0 points [-]

Short version: consider Savage's theorem (fulfilling the conditions by offering Sleeping Beauty a bet along with the question "what day is it?", or by having Sleeping Beauty want to watch a sports game on Monday specifically, etc.). Savages theorem requires your agent to have a preference ordering over outcomes, and have things it can do that lead to different outcomes depending on the state of the world (events), and it states that consistent agents have probabilities over the state of the world.

On both days, Sleeping Beauty satisfies these desiderata. She would prefer to win the bet (or watch the big game), and her actions lead to different outcomes depending on what day it is. She therefore assigns a probability to what day it is.

We do this too - it is physically possible that we've been duplicated, and yet we continue to assign probabilities to what day it is (or whether our favorite sports will be there when we turn on the TV) like normal people, rather than noticing that it is meaningless since we might be simultaneously in different days.

Comment author: Squark 07 September 2015 06:07:04PM 0 points [-]

I disagree with the part "her actions lead to different outcomes depending on what day it is." The way I see it, the "outcome" is the state of the entire multiverse. It doesn't depend on "what day it is" since "it" is undefined. The sleeping beauty's action simultaneously affects the multiverse through several "points of interaction" which are located in different days.

Comment author: Manfred 04 September 2015 06:25:44AM *  1 point [-]

Man, that sleeping beauty digression at the end. I'm so easily aggroed.

I wonder if I should write up a rant or a better attempt at exposition, against this "clearly the problem is underdetermined" position. The correct answer is somewhat difficult and all expositions I've seen (or written!) so far have had big shortcomings. But even if one didn't know the right answer, one should readily conclude that a principled answer exists. We assign things probabilities for fairly deep reasons, reasons undisturbed by trifles like the existence of an amnesia pill.

That aside, good talk :)

Comment author: Squark 06 September 2015 04:55:09PM 0 points [-]

Hi Charlie! Actually I complete agree with Vladimir on this: subjective probabilities are meaningless, meaningful questions are decision theoretic. When the sleeping beauty is asked "what day is it?" the question is meaningless because she is simultaneously in several different days (since identical copies of her are in different days).

[LINK] Vladimir Slepnev talks about logical counterfactuals

7 Squark 03 September 2015 06:29PM

Vladimir Slepnev (aka cousin_it) gives a popular introduction to logical counterfactuals and modal updateless decision theory in the Tel Aviv LessWrong meetup.

[https://www.youtube.com/watch?v=Ad30JlVh4dM&feature=youtu.be]

Comment author: PeterCoin 30 August 2015 07:49:20PM 0 points [-]

I'll dig a little deeper but let me first ask these questions:

What do you define as a coincidence?

Where can I find an explanation of the N 2^{-(K + C)} weighting?

Comment author: Squark 03 September 2015 06:24:16PM 0 points [-]

A "coincidence" is an a priori improbable event in your model that has to happen in order to create a situation containing a "copy" of the observer (which roughly means any agent with a similar utility function and similar decision algorithm).

Imagine two universe clusters in the multiverse: one cluster consists of universe running on fragile physics, another cluster consists of universes running on normal physics. The fragile cluster will contain much less agent-copies than the normal cluster (weighted by probability). Imagine you have to make a decision which produces different utilities depending on whether you are in the fragile cluster or the normal cluster. According to UDT, you have to think as even you are deciding for all copies. In other words, if you make decisions under the assumption you are in the fragile cluster, all copies make decisions under this assumption, if you make decisions under the assumption you are in the normal cluster, all copies make decisions under this assumption. Since the normal cluster is much more "copy-dense", it pays off much more to make decisions as if you are in the normal cluster (since utility is aggregated over the entire multiverse).

The weighting comes from the Solomonoff prior. For example, see the paper by Legg.

Comment author: Squark 30 August 2015 05:13:33PM 2 points [-]

I did a considerable amount of software engineer recruiting during my career. I only called the references at an advanced stage, after an interview. It seems to me that calling references before an interview would take too much of their time (since if everyone did this they would be called very often) and too much of my time (since I think their input would rarely disqualify a candidate at this point). The interview played the most important role in my final decision, but when a reference mentioned something negative which resonated with something that concerned me after the interview, this was often a reason to reject.

Comment author: PeterCoin 26 August 2015 12:06:56AM 0 points [-]

I'm digging into this a little bit, but I'm not following your reasoning. UDT from what I see doesn't mandate the procedure you outline. (perhaps you can show an article where it does) I also don't see how which decision theory is best should play a strong role here.

But anyways I think the heart of your objection seems to be "Fragile universes will be strongly discounted in the expected utility because of the amount of coincidences required to create them". So I'll free admit to not understanding how this discounting process works, but I will note that current theoretical structures (standard model inflation cosmology/string theory) have a large amount of constants that are considered coincidences and also produce a large amount of universes like ours in terms of physical law but different in terms of outcome. I would also note that fragile universe "coincidences" don't seem to me to be more coincidental in character than the fact we happen to live on a planet suitable for life.

Lastly I would also note that at this point we don't have a good H1 or H2.

Comment author: Squark 29 August 2015 07:13:02PM *  1 point [-]

I'm digging into this a little bit, but I'm not following your reasoning. UDT from what I see doesn't mandate the procedure you outline. (perhaps you can show an article where it does) I also don't see how which decision theory is best should play a strong role here.

Unfortunately a lot of the knowledge on UDT is scattered in discussions and it's difficult to locate good references. The UDT point of view is that subjective probabilities are meaningless (the third horn of the anthropic trilemma) thus the only questions it make sense to ask are decision-theoretic questions. Therefore decision theory does play a strong role in any question involving anthropics. See also this.

But anyways I think the heart of your objection seems to be "Fragile universes will be strongly discounted in the expected utility because of the amount of coincidences required to create them". So I'll free admit to not understanding how this discounting process works...

The weight of a hypothesis in the Solomonoff prior equals N 2^{-(K + C)} where K is its Kolomogorov complexity, C is the number of coin flips needed to produce the given observation and N is the number of different coin flip outcomes compatible with the given observation. Your fragile universes have high C and low N.

...but I will note that current theoretical structures (standard model inflation cosmology/string theory) have a large amount of constants that are considered coincidences and also produce a large amount of universes like ours in terms of physical law but different in terms of outcome.

Right. But these are weak points of the theory, not strong points. That is, if we find an equally simple theory which doesn't require these coincidences it will receive substantially higher weight. Anyway your fragile universes have a lot more coincidences than any conventional physical theory.

I would also note that fragile universe "coincidences" don't seem to me to be more coincidental in character than the fact we happen to live on a planet suitable for life.

In principle hypotheses with more planets suitable for life also get higher weight, but the effect levels off when reaching O(1) civilizations per current cosmological horizon because it is offset by the high utility of having the entire future light cone to yourself. This is essentially the anthropic argument for a late filter in the Fermi paradox, and the reason this argument doesn't work in UDT.

Lastly I would also note that at this point we don't have a good H1 or H2.

All of the physical theories we have so far are not fragile, therefore they are vastly superior to any fragile physics you might invent.

Comment author: Squark 23 August 2015 06:28:20PM 1 point [-]

Hi Peter! I suggest you read up on UDT (updateless decision theory). Unfortunately, there is no good comprehensive exposition but see the links in the wiki and IAFF. UDT reasoning leads to discarding "fragile" hypotheses, for the following reason.

According to UDT, if you have two hypotheses H1, H2 consistent with your observations you should reason as if there are two universes Y1 and Y2 s.t. Hi is true in Yi and the decisions you make control the copies of you in both universes. Your goal is to maximize the a priori expectation value of your utility function U where the prior includes the entire level IV multiverse weighted according to complexity (Solomonoff prior). Fragile universes will be strongly discounted in the expected utility because of the amount of coincidences required to create them. Therefore if H1 is "fragile" and H2 isn't, H2 is by far the more important hypothesis unless the complexity difference between them is astronomic.

Comment author: Romashka 16 August 2015 07:46:09PM 0 points [-]

Nothing wrong - if you prove the business scalable. (Which might not be true for many charities out there, but that would not make them inefficient; only the donating as contributing.) I admit I have no experience with free kitchens, though.

Comment author: Squark 17 August 2015 06:18:44PM 0 points [-]

Scalable in what sense? Do you foresee some problem with one kitchen using the hiring model and other kitchens using the volunteer model?

View more: Prev | Next