You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

Jiro comments on A few misconceptions surrounding Roko's basilisk - Less Wrong Discussion

39 Post author: RobbBB 05 October 2015 09:23PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (125)
Tags: roko blackmail roko's basilisk

Comments (125)

You are viewing a single comment's thread. Show more comments above.

Comment author: V_V 14 October 2015 08:40:04AM *  -2 points [-]
  • 1 - Humans can't reliably precommit. Even if they could, precommittment is different than using an "acausal" decision theory. You don't need precommitment to one-box in Newcomb's problem, and the ability to precommit doesn't guarantee by itself that you will one-box. In an adversarial game where the players can precommit and use a causal version of game theory, the one that can precommit first generally wins. E.g. Alice can precommit to ignore Bob's threats, but she has no incentive to do so if Bob already precommitted to ignore Alice's precommitments, and so on. If you allow for "acausal" reasoning, then even having a time advantage doesn't work: if Bob isn't born yet, but Alice predicts that she will be in an adversarial game with Bob and Bob will reason acausally and therefore he will have an incentive to threaten her and ignore her precommitments, then she has an incentive not to make such precommitment.
  • 2 - This implies that the future AI uses a decision theory that two-boxes in Newcomb's problem, contradicting the premise that it one-boxes.
  • 3 - This implies that the future AI will have a deontological rule that says "Don't blackmail" somehow hard-coded in it, contradicting the premise that it will be an utilitarian. Indeed, humans may want to build an AI with such constants, but in order to do so they will have to consider the possibility of blackmail and likely reject utilitarianism, which was the point of Roko's argument.
  • 4 - Shut up and multiply.
Comment author: Jiro 14 October 2015 06:17:11PM *  1 point [-]

Humans can't reliably precommit.

"I precommit to shop at the store with the lowest price within some large distance, even if the cost of the gas and car depreciation to get to a farther store is greater than the savings I get from its lower price. If I do that, stores will have to compete with distant stores based on price, and thus it is more likely that nearby stores will have lower prices. However, this precommitment would only work if I am actually willing to go to the farther store when it has the lowest price even if I lose money".

Miraculously, people do reliably act this way.

Comment author: V_V 14 October 2015 06:39:55PM -1 points [-]

Miraculously, people do reliably act this way.

I doubt it. Reference?

Comment author: CronoDAS 14 October 2015 07:59:21PM 2 points [-]

Mostly because they don't actually notice the cost of gas and car depreciation at the time...

Comment author: Jiro 14 October 2015 08:08:27PM 1 point [-]

You've described the mechanism by which the precommitment happened, not actually disputed whether it happens.

Many "irrational" actions by human beings can be analyzed as precommitment; for instance, wanting to take revenge on people who have hurt you even if the revenge doesn't get you anything.

Powered by Reddit Powered by Reddit