2 30 June 2012 02:46PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Sort By: Best

Comment author: 01 July 2012 02:20:00AM 1 point [-]

Are you familiar with Stuart Armstrong's work on anthropics?

Comment author: 05 July 2012 03:00:41AM 0 points [-]

Obligatory warning of deep flaws.

Comment author: 06 July 2012 02:21:32AM *  0 points [-]

You provide very little information. I'm not even sure what you disagree with exactly. If it would be inconvenient for you to explain your disagreement that's fine, but I'm didn't update much on your comment. If you want to give me a bit more information about your state of mind, you can tell me how familiar you are with UDT and whether you think it is on the right path towards solving anthropics.

Comment author: 06 July 2012 05:33:58AM *  1 point [-]

Yeah, sorry about that. The basic idea is that by providing multiple answers, the proposal has immediately given up on getting the same answer as an expected utility maximizer. This is a perfectly fine thing to do if probabilities are impossible to assign, and so maximizing expected utility breaks down. But probability does not in fact break down when confronted with anthropic situations, so picking from N answers just gives you at least an (N-1)/N chance of being wrong.

Comment author: 11 July 2012 12:28:29AM 1 point [-]

Expected utility does break down in the presence of indexical uncertainty though; if there are multiple agents with exactly your observations, it is important to take into account that your decision is the one they will all make. Psy-Kosh's non-anthropic problem deals with this sort of thing, though it also points out that such correlation between agents can exist even without indexical uncertainty, which is irrelevant here.

I'm not sure what the N answers that you are talking about are. The different solutions in Stuart's paper refer to agents with different utility functions. Changing the utility function usually does change the optimal course of action.

Comment author: 11 July 2012 03:24:27AM *  1 point [-]

Psy-Kosh's non-anthropic problem is just regular uncertainty. The experimenters flipped a coin, and you don't know if the coin is heads or tails. The collective decision making then runs you into trouble. I can't think of any cases with indexical uncertainty but no collective decision making that run into similar trouble - in the Sleeping Beauty problem at least the long-term frequency of events is exactly the same as the thing you plug into the utility function to maximize average reward, unlike in the non-anthropic problem. Do you have an example you could give?

EDIT: Oh, I realized one myself - the absent-minded driver problem. In that problem, if you assign utility to the driver at the first intersection - rather than just the driver who makes it to an exit - you end up double-counting and getting the wrong answer. In a way it's collective decision-making with yourself - you're trying to take into account how past-you affected present-you, and how present-you will affect future-you, but the simple-seeming way is wrong. In fact, we could rejigger the problem so it's a two-person, non anthropic problem! Then if we do the reverse transform on Psy-kosh's problem, maybe we could see something interesting... Update forthcoming, but the basic idea seems to be that the problem is when you're cooperating with someone else, even yourself, and are unsure who's filling what role. So you're pretty much right.

The objects in Stuart's paper are decision procedures, but do not involve utility directly (though it is a theorem that you can find a utility function that gives anything). Utility functions have to use a probability before you get a decision out, but these decision procedures don't. Moreover, he uses terms like "average utilitarian" to refer to the operations of the decision procedure (averages individual utilities together), rather than the properties of the hypothetical corresponding utility function.

What's happening is that he's taking an individual utility function and a decision procedure, and saying that together these specify what happens. And I'm saying that this is an over-specified problem.

Comment author: 12 July 2012 03:32:08PM 0 points [-]

Then if we do the reverse transform on Psy-kosh's problem, maybe we could see something interesting...

That's what I was originally trying to suggest, but it seems I was unclear. The absent-minded driver is a simpler example anyways, and does deal with exactly the kind of breakdown of expected utility I was referring to.

What's happening is that he's taking an individual utility function and a decision procedure, and saying that together these specify what happens. And I'm saying that this is an over-specified problem.

Each decision procedure is derived from a utility function. From the paper:

Anthropic Decision Theory (ADT) An agent should first find all the decisions linked with their own. Then they should maximise expected utility, acting as if they simultaneously controlled the outcomes of all linked decisions, and using the objective (non-anthropic) probabilities of the various worlds.

This fully specifies a decision procedure given a utility function. There is no second constraint taking the utility function into account again, so it is not overspecified.

Comment author: 12 July 2012 04:09:37PM 1 point [-]

Simpler? Hm. Well, I'm still thinking about that one.

Anyhow, by over-specified I mean that ADT and conventional expected-utility maximization (which I implicitly assumed to come with the utility function) can give different answers. For example, in a non-cooperative problem like copying someone either once or 10^9 times, and then giving the copy a candybar if it can correctly guess how many of them there are. The utility function already gives an answer, and no desiderata are given that show why that's wrong - in fact, it's one of the multiple possible answers laid out.

Comment author: 12 July 2012 09:53:00PM *  2 points [-]

Simpler in that you don't need to transform it before it is useful here.

Standard expected utility maximization requires a probability distribution, but the problem is that in anthropic scenarios it is not obvious what the correct distribution is and how to correctly update it. ADT uses the prior distribution before 'observing one's own existence', so it circumvents the need to preform anthropic updates.

I'm not sure which solution to your candybar problem you think is correct because I am not sure which probability distribution you think is correct, but all the solutions in the paper that disagree with yours actually are what you would want to precommit to given the associated utility function and are therefore correct.

Comment author: 13 July 2012 05:33:20AM *  1 point [-]

Standard expected utility maximization requires a probability distribution, but the problem is that is anthropic scenarios it is not obvious what the correct distribution is and how to correctly update it.

If it was solved in a way that made it obvious for, say, the Sleeping Beauty problem, would that then be the right way to do it?

all the solutions in the paper that disagree with yours actually are what you would want to precommit to given the associated utility function and are therefore correct.

I think you're just making up utility functions here - is a real utility function (that is, a function of the state of the world) ever calculated in the paper, other than the use of the individual utility function? And we're talking about regular ol' utility functions, why are ADT's decisions necessarily invariant under changing time-like uncertainty (normal sleeping beauty problem) to space-like uncertainty (sleeping beauty problem with duplicates)?