I think I'm using "anthropic" in a way consistent with the end of the first paragraph of Fundamentals of kicking anthropic butt (to refer to situations in which agents get duplicated and/or there is some uncertainty about what agent an agent is). If there's a more appropriate word then I'd appreciate knowing what it is.

My first objection is already contained in Vladimir_Nesov's comment: it seems like in general anthropic problems should be phrased entirely as decision problems and not as problems involving the assignment of odds. For example, Sleeping Beauty can be turned into two decision problems: one in which Sleeping Beauty is trying to maximize the expected number of times she is right about the coin flip, and one in which Sleeping Beauty is trying to maximize the probability that she is right about the coin flip. In the first case, Sleeping Beauty's optimal strategy is to guess tails, whereas in the second case it doesn't matter what she guesses. In a problem where there's no anthropic funniness, there's no difference between trying to maximize the expected number of times you're right and trying to maximize the probability that you're right, but with anthropic funniness there is.

My second objection is that I don't understand how an agent could be convinced of the truth of a sufficiently bizarre premise. (I have the same issue with Pascal's mugging, torture vs. dust specks, and Newcomb's problem.) In this particular case, I don't understand how I could be convinced that another agent really has the capacity to perfectly simulate me. This seems like exactly the kind of thing that agents would be incentivized to lie about in order to trick me.

Comment author: Wei_Dai 19 January 2013 12:56:50AM 3 points [-]

My second objection is that I don't understand how an agent could be convinced of the truth of a sufficiently bizarre premise. (I have the same issue with Pascal's mugging, torture vs. dust specks, and Newcomb's problem.) In this particular case, I don't understand how I could be convinced that another agent really has the capacity to perfectly simulate me. This seems like exactly the kind of thing that agents would be incentivized to lie about in order to trick me.

You may eventually obtain the capacity to perfectly simulate yourself, in which case you'll run into similar issues. I used Omega in a scenario a couple of years ago that's somewhat similar to the OP's, but really Omega is just a shortcut for establishing a "clean" scenario that's relatively free of distractions so we can concentrate on one specific problem at a time. There is a danger of using Omega to construct scenarios that have no real-world relevance, and that's something that we should keep in mind, but I think it's not the case in the examples you gave.

Comment author: ESRogs 11 January 2013 02:35:30AM 2 points [-]

How would you characterize your issue with Pascal's mugging? The dilemma is not supposed to require being convinced of the truth of the proposition, just assigning it a non-zero probability.

Comment author: Qiaochu_Yuan 11 January 2013 02:55:43AM 3 points [-]

Hmm. You're right. Upon reflection, I don't have a coherent rejection of Pascal's mugging yet.

Comment author: ESRogs 11 January 2013 04:13:22AM 1 point [-]

Gotcha. Your posts have seemed pretty thoughtful so far so I was surprised by / curious about that comment. :)

Comment author: MugaSofer comment score below threshold [+] (13 children)

Comment author: Qiaochu_Yuan 11 January 2013 08:18:00PM 1 point [-]

How then, do you assign the odds?

You don't.

You believe Omega because it's Omega, who always tells the truth and has access to godlike power.

How does any particular agent go about convincing me that it's Omega?

Comment author: MugaSofer 13 January 2013 10:28:40AM -2 points [-]

How then, do you assign the odds?

You don't.

So ... you don't know what the odds are, but you know how to act anyway? I notice that I am confused.

it's Omega, who always tells the truth and has access to godlike power.

How does any particular agent go about convincing me that it's Omega?

Assuming that there is only one godlike agent known or predicted in the environment, that Omega is a known feature of the environment, and that you have no reason to believe that e.g. you are hallucinating, then presumably all Omega needs to do is demonstrate his godlike powers - by predicting your every action ahead of time, say, or turning the sky green with purple spots.

Comment author: Qiaochu_Yuan 13 January 2013 12:22:35PM 0 points [-]

So ... you don't know what the odds are, but you know how to act anyway? I notice that I am confused.

In an anthropic situation, I don't think it makes sense to assign odds to statements like "I will see X" because the meaning of the term "I" becomes unclear. (For example, I don't think it makes sense for Sleeping Beauty to assign odds to statements like "I will see heads.") I can still assign odds to statements like "exactly fifteen copies of me will see X" by reasoning about what I currently expect my copies to see, given what I know about how I'll be copied, and using those odds I can still make decisions.

Assuming that there is only one godlike agent known or predicted in the environment, that Omega is a known feature of the environment, and that you have no reason to believe that e.g. you are hallucinating, then presumably all Omega needs to do is demonstrate his godlike powers - by predicting your every action ahead of time, say, or turning the sky green with purple spots.

Omega needs to both always tell the truth and have access to godlike power. How does Omega prove to me that it always tells the truth?

Comment author: MugaSofer comment score below threshold [+] (8 children)

Comment author: Qiaochu_Yuan 13 January 2013 05:19:25PM 0 points [-]

What do you think Alice should choose?

There's nothing to choose until you've specified a decision problem.

It is a known feature of the environment that people are regularly ambushed by an agent, calling itself Omega, which has never yet been known to lie and has access to godlike power.

An agent with godlike power can manufacture evidence that it has any other traits it wants, so observing such evidence isn't actually evidence that it has those traits.

Comment author: MugaSofer comment score below threshold [+] (6 children)

Comment author: Decius 12 January 2013 02:27:56AM -1 points [-]

How does any particular agent go about convincing me that it's Omega?

I don't know, but Omega does. Probably by demonstrating the ability to do something such that you believe the chance that it could be faked are epsilon^2, where epsilon is your prior belief that a given agent could have godlike powers.

MugaSofer comments on DRAFT:Ethical Zombies - A Post On Reality-Fluid - Less Wrong

Article Navigation

Comments (116)

Nearest Meetups

Virtual Study Room

Recent Comments

Recent Posts

Recent Rationality Quotes

Recent Wiki Edits

Top Contributors, 30 Days

Recent Karma Awards

Comments (116)

Top Contributors, 30 Days

You'll need to login or register to do that

(Don't worry, it only takes a few seconds)

Create

Login