Roko comments on Counterfactual Mugging - Less Wrong

52 Post author: Vladimir_Nesov 19 March 2009 06:08AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (257)
Tags: decision_theory commitment counterfactual_mugging

Comments (257)

You are viewing a single comment's thread. Show more comments above.

Comment deleted 19 March 2009 01:03:34PM [-]
Comment author: Nebu 19 March 2009 06:19:44PM 2 points [-]

A (quasi)rational agent with access to genuine randomness (such as a human)

Whaddaya mean humans are rational agents with access to genuine randomness? That's what we're arguing about in the first place!

A superintelligence could almost perfectly predict the probability distribution over my actions, but by quantum entanglement it would not be able to predict my actual actions.

Perhaps Omega is entangled with your brain such that in all the worlds in which you would choose to one-box, he would predict that you one-box, and all the worlds in which you would choose to two-box, he would predict that you two-box?

Comment author: Eliezer_Yudkowsky 19 March 2009 07:32:52PM 3 points [-]

In the original formulation, if Omega expects you to flip a coin, he leaves box B empty.

Comment author: Kingreaper 23 June 2010 01:39:07PM *  -1 points [-]

Imagine knowing with certainty that your actions can be predicted perfectly by the guy next door, even taking into account that you are trying to be hard to predict?

You wouldn't know this with certainty* because it wouldn't be true.

(*unless you were delusional)

The guy next door is on roughly your mental level. Thus, the guy next door can't predict your actions perfectly, because he can't run a perfect simulation of your mind that's faster than you. He doesn't have the capacity.

And he certainly doesn't have the capacity to simulate the environment, including other people, while doing so.

A (quasi)rational agent with access to genuine randomness (such as a human) is a different matter.

Humans may or may not generally have access to genuine randomness.

It's as yet unknown whether we even have run on quantum randomness; and its also unprovable that quantum randomness is actually genuine randomness, and not just based on effects we don't yet understand, as so many other types of randomness have been.

Comment author: thomblake 23 June 2010 02:56:46PM 1 point [-]

You wouldn't know this with certainty* because it wouldn't be true.

You're not taking this in the least convenient possible world. Surely it's not impossible in principle that your neighbor can simulate you and your environment. Perhaps your neighbor is superintelligent?

Comment author: Kingreaper 23 June 2010 03:21:09PM *  1 point [-]

It's ALSO not impossible in principle in the real world. A superintelligent entity could, in principle, perfectly predict my actions. Remember, in the Least Convenient Possible World quantum "randomness" isn't random.

As such, this ISN'T a fundamental difference between humans and "such beings". Which was all I set out to demonstrate.

I was using the "most plausible world" on the basis that it seemed pretty clear that that was the one Roko intended. (Where your neighbour isn't in fact Yahweh in disguise). EDIT: Probably should specify worlds for things in this kind of environment. Thanks, the critical environment here is helping me think about how I think/argue.

Powered by Reddit Powered by Reddit