LESSWRONG
LW

Comment Permalink

eternal_neophyte8y20

This is the first time I've heard of this dilemma (so this post is really just thinking aloud). It seems to me that trade usually doesn't require agents to engage in deep modeling of each other's behaviour. If I go down to the market place and offer the man at the stall £5 for a pair of shoes, and he declines and I walk away - the furthest thing from my mind is trying to step through my model of human behaviour to figure out how to persuade him to accept. I had a simple model - to wit that the £5 was sufficient incentive to effect the trade - and when that model turned out false I just abandoned negotiations without trying to calculate the incentive effects of doing so.

This isn't to say that everything involving deep modeling of human behaviour is necessarily an instance of extortion, though the converse would seem to hold ( every act of extortion involves some higher-order modeling between extorter and victim ). However, extortion usually involves the extorter trying to increase the cost of select outcomes above what they would be had the extorter not explicitly acted to increase them, which is why deep modeling of the victim is required. Unless my costly, deep model of my trading partner is paying rent to me ( with respect to a given episode of negotiation ) only in some way that does not involve allowing me to increase the cost of a certain set of outcomes to him in some negotiation, I am probably engaging in extortion.

If I walk way from a market stall with the intent of provoking the seller into lowering his price - I'm not increasing the cost of any outcome to him. The cost of me walking away is a constant. So in this case my model of his behaviour is not aimed at increasing the cost of any outcome to him - I'm effectively simply placing a bet. If I threaten to break his legs if he refuses the sale, that's placing a bet on a rigged game.

Stuart_Armstrong8y10

What of companies that spend millions analysing markets before setting their prices? That seems to involve deep modelling, yet is canonically seen as trade.

6

[Stub] Extortion and Pascal's wager

by Stuart_Armstrong

26th Apr 2017

1 min read

6

The premises of Pascal's wager are normally presented as abstract facts about the universe - there happens to (maybe) be a god, who happens to have set up the afterlife for the suffering of unbelievers.

But, assuming we ever manage to distinguish trade from extortion, this seems a situation of classical extortion. So if god follows a timeless decision theory - and what other kind of decision theory would it follow? - the correct answer would seem to be to reject the whole deal out of hand, even if you assume god exists.

Or, in other words, respond to a god that offers you heaven, but ignore one that threatens you with hell.

6

New Comment

29 comments, sorted by

Click to highlight new comments since: Today at 9:39 AM

[-]entirelyuseless8y20

One problem with this is that the "abstract facts" interpretation is actually more reasonable than the idea that a good God exists and threatens to torture some people for eternity. This is why modern Christians themselves tend more towards the abstract fact interpretation of hell, and are less willing to talk about it in terms of punishment.

In the future, as it becomes more and more clear how much determination of our actions comes from our brains, Christianity will move even more in this direction. That is more reasonable than the opposite, and so it is also a more reasonable interpretation when you are considering the wager.

There is another factor here too. Calling the threat "extortion" implies that such a threat would be evil. I agree that it would be, as a matter of fact, but someone considering the probability that "a revealed religion is true" should not be considering "a revealed religion is true, and God is evil," but "a revealed religion is true, and God is good despite all appearances." Obviously the claim that God is good is an essential part of basically every purportedly revealed religion, so if you consider the hypothesis that God is evil, you are not considering the possibility of those religions at all. So the only way of actually considering the hypothesis implies the denial that the threat counts as extortion.

Obviously the claim that God is good is an essential part of basically every purportedly revealed religion

Doesn't seem so obvious to me. I suspect that "God is good" is an invention of the recent few thousand years. Previously, gods were simply supernatural beings, doing some random things, sometimes threatening people "I will punish you if you do X" or offering them trade "if you do X, I will (maybe) give you Y".

Even the relatively recent Gods often seem only "good" in the sense "yeah, he does some unpleasant things, but if you call him evil, he will torture you, so it is better to call him good". Essentially, a supernatural Stalin with omniscient secret service; of course you would only speak positively about him, regardless of what he does.

[-]entirelyuseless8y10

Even the relatively recent Gods often seem only "good" in the sense "yeah, he does some unpleasant things, but if you call him evil, he will torture you, so it is better to call him good". Essentially, a supernatural Stalin with omniscient secret service; of course you would only speak positively about him, regardless of what he does.

It is fine to say that religious claims are false; it is not fine to substitute strawman claims and say they are the real claims.

The notion that God might be evil is not a mere "strawman"; it is in fact a feature of some actual religions, notably Gnostic Christianity. Of course, the Book of Job in the Hebrew Bible seems to aim at refuting this very argument, since it freely admits that God often appears to be evil from our point of view; but since he is infinitely wise and knows so many things which are entirely beyond our understanding, and has done plenty of nice things for us besides (such as laying the foundations of our very world), how can we argue against his goodness?

(All this in response to the anguished questioning of a man who, in-story, has had almost everything taken away from him by -- or at least with the express permission of -- God ... for a bet.)

And a bet with Satan, no less.

Yup. (Though that bit isn't quite as bad as it sounds: "Satan" in Job isn't at all the same figure as in present-day Christianity.)

[-]entirelyuseless8y00

Viliam's account of a "supernatural Stalin" was in fact a strawman. It is not even equivalent to the evil God in gnosticism. The gnostics thought you were supposed to reject that God and not do what he says; you definitely would not say that he was good. In contrast, no one believes Viliam's theory, and so it was definitely a strawman.

It is also strawmanning, although less indefensibly, to imply that modern religious beliefs are equivalent to ancient extinct ones.

[-]eternal_neophyte8y00

My own reading of Job was not that god's goodness is undeniable, it's that god really needs nothing from us and is entirely indifferent to human beings choosing to damn themselves or not, in contradiction to "your God is a jealous God".

If you have sinned, what do you accomplish against him? And if your transgressions are multiplied, what do you do to him? If you are righteous, what do you give to him? Or what does he receive from your hand? Your wickedness concerns a man like yourself, and your righteousness a son of man.

This seems to me like the most sane piece of theological reasoning I've found in any religious text whatever - casting God as an entirely amotivational agent ( which is strangely in contradiction to the premise of the story of Job ).

god really needs nothing from us and is entirely indifferent to human beings choosing to damn themselves or not

And then God says "j/k, just kidding" and does the whole New Testament thing :-)

[-]eternal_neophyte8y00

God says "j/k, just kidding"

Either God, Jesus or St. Paul - that all depends entirely on which sect you ask.

Got to be someone from the Holy Trinity -- Paul isn't going to cut it.

[-]eternal_neophyte8y00

Paul isn't going to cut it

Paul might cut it if you're Thomas Jeffson: https://en.wikipedia.org/wiki/Jefferson_Bible "Paul was the first corrupter of the doctrines of Jesus."

Not for the change of mind from "I completely don't care about humans" to "I'll make my only Son a human (to start with) and let other humans crucify him so that they could wash off the original sin".

[-]eternal_neophyte8y00

It's not smoking-gun obvious to me that this second formulation is what the pre-Pauline Christians believed in. Jesus's divinity certainly wasn't settled even after Paul. Consider for example the Arian "heresy".

My reading of Job is that Leviathan is more awesome than humans, and Job is forced to admit this, therefore God optimizes this world for Leviathan instead of humans. It's not that humans are completely irrelevant; but they are merely a rounding error compared with Leviathan, the utility monster.

[-]eternal_neophyte8y00

therefore God optimizes this world for Leviathan

?

I dunno if there's that much of a difference. Either way (eternal bliss/torment) it would be a moral consideration that swamps all others. Surely you gotta treat them both the same way, yeah?

[-]Stuart_Armstrong8y20

This line of thinking illustrates the difficulty of distinguishing extortion from trade possibilities.

assuming we ever manage to distinguish trade from extortion

How is that coming? The previous thread went nowhere. I am of the opinion (lightly and willing to change) that the distinction is in the intent of the causal agent and in our VERY sketchy understanding of moral actions with and without trade possibilities. If something is permitted but not required to do (or not to do), then being paid to do (or refrain) from it is also permitted.

Also, it's extremely unclear how you go from "this is acausal extortion" to "reject the deal". You wouldn't say "two-box if there are negative values involved, one-box if it's positive", would you? If the basilisk were real and you couldn't prevent it, I think sane behavior (under any decision theory that supports this style of question) would be to cooperate.

[-]Stuart_Armstrong8y00

How is that coming? The previous thread went nowhere.

Still nowhere :-(

If you want to pursue the subject, I recommend googling Law Review articles about blackmail -- the lawyers have to deal with this problem as a practical matter. Ignore the ones which dive deep into case law details, but look at ones which take the philosophical start-from-first-principles approach.

[-]Stuart_Armstrong8y00

Are you familiar with that literature?

I scanned through some of it long time ago, so I know it exists but, unfortunately, cannot offer links.

[-]eternal_neophyte8y20

This is the first time I've heard of this dilemma (so this post is really just thinking aloud). It seems to me that trade usually doesn't require agents to engage in deep modeling of each other's behaviour. If I go down to the market place and offer the man at the stall £5 for a pair of shoes, and he declines and I walk away - the furthest thing from my mind is trying to step through my model of human behaviour to figure out how to persuade him to accept. I had a simple model - to wit that the £5 was sufficient incentive to effect the trade - and when that model turned out false I just abandoned negotiations without trying to calculate the incentive effects of doing so.

This isn't to say that everything involving deep modeling of human behaviour is necessarily an instance of extortion, though the converse would seem to hold ( every act of extortion involves some higher-order modeling between extorter and victim ). However, extortion usually involves the extorter trying to increase the cost of select outcomes above what they would be had the extorter not explicitly acted to increase them, which is why deep modeling of the victim is required. Unless my costly, deep model of my trading partner is paying rent to me ( with respect to a given episode of negotiation ) only in some way that does not involve allowing me to increase the cost of a certain set of outcomes to him in some negotiation, I am probably engaging in extortion.

If I walk way from a market stall with the intent of provoking the seller into lowering his price - I'm not increasing the cost of any outcome to him. The cost of me walking away is a constant. So in this case my model of his behaviour is not aimed at increasing the cost of any outcome to him - I'm effectively simply placing a bet. If I threaten to break his legs if he refuses the sale, that's placing a bet on a rigged game.

[-]Stuart_Armstrong8y10

What of companies that spend millions analysing markets before setting their prices? That seems to involve deep modelling, yet is canonically seen as trade.

[-]eternal_neophyte8y00

They usually don't have any way to leverage their models to increase the cost of not buying their product or service though; so such a situation is still missing at least one criterion.

There is a complication involved since its possible to increase the cost to others of not doing business with you in "fair" ways. E.g. the invention of the fax machine reduced effective demand for message boys to run between office buildings, hence increasing their cost and the operating costs of anyone who refused to buy a fax machine.

Though I don't believe any company long held a monopoly on the fax market, if a company did establish such a monopoly in order to control prices that again may be construed as extortion.

They usually don't have any way to leverage their models to increase the cost of not buying their product or service though; so such a situation is still missing at least one criterion.

Modern social networks and messaging networks would seem to be a strong counterexample. Any software with both network effects and intentional lock-in mechanisms, really.

And honestly, calling such products a blend of extortion and trade seems intuitively about right.

To try to get at the extortion / trade distinction a bit better:

Schelling gives us definitions of promises and threats, and also observes there are things that are a blend of the two. The blend is actually fairly common! I expect there's something analogous with extortion and trade: you can probably come up with pure examples of both, but in practice a lot of examples will be a blend. And a lot of the 'things we want to allow' will look like 'mostly trade with a dash of extortion' or 'mostly trade but both sides also seem to be doing some extortion'.

Any software with both network effects and intentional lock-in mechanisms, really.

The cost of not buying is not the same thing as the cost of switching.

More from Stuart_Armstrong

Curated and popular this week