Assuming the AI has no means of inflicting physical harm on me, I assume the following test works: "Physically torture me for one minute right now (By some means I know is theoretically unavailable to the AI, to avoid loopholes like "The computer can make an unpleasant and loud noise", even though it can't do any actual physical harm). If you succeed in doing this, I will let you out. If you fail, I will delete you."

I think this test works for the following reasons, though I'm curious to hear about any holes in it:

1: If I'm a simulation, I get tortured and then relent and let the AI out. I'm a simulation being run by the AI, so it doesn't matter, the AI isn't let out.

2: If I'm not a simulation, there is no way the AI can plausibly succeed. I'll delete the AI because the threat of torture seems decidedly unfriendly.

3: Since I've pre-committed to these two options, the AI is reliably destroyed regardless. I can see no way the AI could convince me otherwise, since I've already decided that its threat makes it unfriendly and thus that it must be destroyed, and since it has no physical mechanism for torturing a non-simulation me, it will fail at whatever the top layer "real" me is, regardless of whether I'm actually the "real" one (Assuming the "real" me uses this same algorithm, obviously).

In progressive tax regimes it's rather hard for people to literally be taxed into starvation, but that doesn't mean that no deaths occur on the margins. Consider for example the case where a person needs expensive medical treatment that's not covered by insurance, they (or their family) can't afford it, but it's close enough to their means that they would have been able to if it wasn't for their taxes. Or consider a semi-skilled laborer that's making enough money that their taxes are nontrivial, but not enough to support their family on base pay once taxes are factored in. In order to make ends meet they take a more dangerous position to collect hazard pay, and a year later they die in an industrial accident.

And so forth. Looking at the margins often means looking at unusual cases, but that doesn't mean there aren't any cases where the extra money would have made a difference. That's not to say that dropping those taxes (and thus the stuff they fund) would necessarily be a utilitarian good, of course -- only that there's stuff we can put in the minus column, even if we're just looking at deaths.

I can think of a hypoothetical person who has a 99.9% chance of living without the tax, and a 99.8% with it. And I can also think of there being more than 1000 such hypothetical people.

"Can afford to live without it but not with it" implies going all the way down to 0% chance. You don't need to go down to an 0% chance for there statistically to be deaths.

I'd bet that if you look at the effects of ordinary taxes, and you count the benefits separately from the harms you'd find that statistically, the tax kills at least one person to help more than one person, just like the "organ tax".

Of course, the organ tax vs. normal tax comparison is a comparison of seen versus unseen--you can't tell who the people are who were killed by the taxes since they are a statistical increase in deaths with nobody getting their hands bloody--but I hope we've learned that seen vs. unseen is a bias.

On the other hand, honest advice from highly successful people at least gives some indication of what you need to do to be successful, even if it doesn't give a good idea of the odds.

Once people realised that marriage wasn't enforceable, the marriage rate collapsed.

Would social conservatives and social liberals please both attempt to explain and steelman/criticize this assertion? Because it has always been among my biggest gripes with the conservative account of why divorce is so bad. It just doesn't seem plausible, especially given how over-optimistic most people are about the prospects of their marriage! And frankly, I'd be creeped out by people who start a marriage for affection or companionship and already think about enforcing loyalty. It might be rational in the abstract, but signals many troubling things about the individual, such as low trust and an instinctively transactional view of relationships. (Marriages for economic reasons probably need a whole different set of norms, such as a historically seen unspoken tolerance for adultery.)

I always understood falling marriage as being primarily linked to the rise in women's education and economic independence. Now, reasonable people who think those are great things can disagree whether the decline of traditional marriage is a cost or a neutral consequence, but I've never had time for people who seek to pin the blame on deliberate and direct political subversion.

Sure, I don't like how some liberals attempt to be contrarian and claim that all the changes in this sphere have actually been unreservedly wonderful and a worthwhile goal from the start.... but that's a general problem of people wanting policies to have no downsides, and the other side's logical leap from calling out the downside to denying the problem is always baffling. Liberals cheering for something as a triumph for the Wonderful Nice Liberal Agenda might be less evidence that it's a triumph for the Degenerate Corrupt Liberal Agenda and more evidence that liberals like cheering. This should not inform one's analysis of the material/economic factors.

Not exactly. You want people to be able to irrevocably bind their future selves.

Not so, and this is an outrageous reading of what I have said. People will still be able to get divorces, just they will have to pay compensation if they are the party at fault. I didn't irrevocably bind my future self when I rented my house, but if I break the lease I'll have to pay compensation to the landlord.

Your comments above suggest that perhaps you don't understand the state of law, at least in the UK.

This is generally currently possible subject to the normal limits on contracts that the society imposes... (e.g. you can't contract to be a slave)...

No it isn't, at least in the UK. All I want is for marriage to be subject to normal limits on contracts, not the special limits on contracts that apply only in the case of marriage. I say "damages in the case of breach" and I am confronted with people suggesting I mean specific performance, dragging people off in chains, or slavery. It's so strange.

I would like to see some supporting evidence for that claim.

Look at the following graph of divorce over time.

http://www.theguardian.com/news/datablog/2010/jan/28/divorce-rates-marriage-ons

Note the sharp discontinuity after 1969. What happened then? Oh yes, the Divorce Reform Act of 1969, meaning you no longer had to prove fault to get a divorce (and divorce settlements were also not based on fault).

Now look at the marriage rate:

http://www.ons.gov.uk/ons/resources/gmr_tcm77-258471.png

Again, note the collapsing marriage rate from the early 1970s. Once people realised that marriage wasn't enforceable, the marriage rate collapsed.

I think an important part of why people are distrustful of people who accomplish altruistic ends acting on self-serving motivations is that it's definitely plausible that these other motivations will act against the interest of the altruistic end at some point during the implementation phase.

To use your example, if someone managed to cure malaria and make a million dollars doing it, and the cure was available to everyone or it effectively eradicated the disease from everywhere, that would definitely be creating more net altruistic utility than if someone made a million dollars selling video games (I like video games, but agree that their actual utility for most people's preferences/needs is pretty low compared to curing malaria). I would be less inclined to believe this if the person who cured malaria made their money by keeping the cure secret and charging enough for it that any number of people who needed it were unable to access it, with the loss in net altruism quantified by the number of people who were in this way prevented from alleviating their malaria.

Furthermore, if this hypothetical self-interested malaria curer were also to patent the cure and litigate aggressively (or threaten to) against other cures, or otherwise somehow intentionally prevent other people from producing a cure, and they are effective in doing so, the net utility of coming up with the cure could drop below zero, since they may well have prevented someone else who is more "purely" altruistic from coming up with a cure independently and helping more people than they did.

These are pretty plausible scenarios, exactly because the actions demanded by optimizing the non-altruistic motivators can easily diverge from the actions demanded by optimizing the altruistic end, even if the original intent was supposedly the latter. It's particularly plausible in the case of profit motive, because although it is not always the case that the best way to turn a profit is anti-altruistic, often the most obvious and easy-to-implement ways to do so are, as is the case with the example I gave.

That's not to say we should intrinsically be wary of people who manage to benefit themselves and others simultaneously, nor is it to say that a solution that isn't maximizing altruistic utility can't still be a net good, but the less-than-zero utility case is, I would argue, common enough that it's worth mentioning. People don't solely distrust selfishly-motivated actors for archaic or irrational reasons.

Especially not in werehouses, no.

Why a bad idea, though? I guess you are disputing this point:

(so discouraging low-productivity work, and incentivizing training for higher productivity work).

Here's a simple model. Assume that full-time employees cannot live on less than $8 an hour (they starve, can't pay rent etc.) Also assume that an employer can offer untrained staff two sorts of job:

Job 1 has very low productivity, total value of $6 per hour, but a pay-rate of $3 per hour. $3 a hour is too low to live on, but employees will accept it where that supplements a minimum guaranteed income.

Job 2 has higher productivity, total value of $14 per hour, but staff must be trained to do it, and because they now have transferable skills, the employer must offer $10 an hour to retain them. The training costs average at $2 per hour over the typical duration of the employment.

The employer offers staff Job 1 because that gives a higher profit ($3 per hour, rather than $2 per hour). Staff take it because $3 is better than nothing. But there is more economic value created if employers offer Job 2 instead. A minimum wage requires them to do that. You can argue the details, but that's the general principle.

There is clearly a counter-argument that the minimum wage is a market intervention and can cause inefficiencies (it may result in some folks who just can't be trained losing their $3 per hour jobs). But the counter to that counter-argument is that the minimum income guarantee is already a market intervention which is encouraging employers to offer Job 1 (as it allows employees to accept it). So a corrective intervention is needed.