= 27d567bc2d48d9f40e9ccc20ea370b15)
ArisKatsaris comments on The Aliens have Landed! - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (156)
That's the only sane reason you believe can exist for caring about distant people at all? That you can potentially travel to them?
So if you're a paraplegic , who doesn't want to travel anywhere, can't travel anywhere, and know you'll die in two weeks anyway. You get a choice to push a button or not push it. If you push it you get 1 dollar right now, but 1 billion Chinese people will die horrible deaths in two weeks, after your own death.
Are you saying that the ONLY "sane" choice is to push the button, because you can use the dollar to buy bubblegum or something, while there'll never be a consequence on you for having a billion Chinese die horrible deaths after your own death?
If so, your definition of sanity isn't the definition most people have. You're talking about the concept commonly called "selfishness", not "sanity".
Fine. Explain to me why Fred shouldn't exterminate his species, or tell me that he should.
The extortion aspect isn't essential. Fred could have been manipulated by true claims about making simulated people super happy.
ETA: At one point this comment had downvotes but no reply, but when I complained that that wasn't a rational discussion, someone actually replied. LessWrong is doing what it's supposed to do. Thanks people for making it and participating in it.
I would give in to the alien demands in that situation, assuming we 'least convenient possible world' away all externalities (the aliens might not keep their promise, there might be quadrillions of sentient beings in other species who we could save by stopping these aliens).
The way the story is told makes it easy for us to put ourselves in the shoes of Fred, Thud or anyone else on earth, and hard to put ourselves in the shoes of the simulations, faceless masses with no salient personality traits beyond foolishness. This combination brings out the scope insensitivity in people.
A better way to tell the story would be to spend 1000 times as many words describing the point of view of the simulations as that of the people on earth. I wonder how ridiculous giving in would seem then.
It's good to have a variety of opinions on the mass-suicide issue. Thanks for posting.
IMO scope insensitivity is a good thing. If you don't have scope insensitivity, then you have an unbounded utility function, and in the event you actually have to compute your utility function, you can't effectively make decisions because the expected utility of your actions is a divergent sum. See Peter de Blanc's paper and subsequent discussion here. If your values incorporate the sum of a divergent series, what happens in practice is that your opinion at the moment varies depending on whichever part of the divergent sum you're paying attention to at the moment. Being vulnerable to Pascal's Wager, Pascal's Mugging, and having Fred's mass-suicide preference in the story are all symptomatic of having unbounded utility functions. Shut Up and Multiply is just wrong, if you start from the assumption that scope insensitivity is bad.
ETA: I should probably have said "Shut Up and Multiply" is just wrong, if one of the things you're multiplying by is the number of individuals affected.
If you truly believe this then I think we really do have no common ground. Scope insensitivity experiments have found that people will pay more to save one child than to save eight. Effectively, it will make you kill seven children and then make you pay for the privilege. It is, IMO, morally indefensible and probably responsible for a great deal of the pain and suffering going on as we speak.
A central tenet of the theory of rationality, as I understand it, is that decision theory, the computational methods used and even method of updating, are all up for grabs if a more convenient alternative is found, but the utility function is not.
My utility function is what I want. Rationality exists to help us achieve what we want, if it is 'rational' for me to change my utility function, then this 'rationality' is a spectacular failure and I will abandon it for an 'irrational alternative'.
What you point out may be a problem, but the solution lies in decision theory or algorithmics, not in paying to kill children.
I agree that those people are confused in some way I do not understand.
Agreed. We need some way to reconcile that with the aforementioned result about scope-insensitive people who would pay the wrong price for saving one child verses saving eight.
True. I think your process for introspecting your utility function is wrong, and I think the procedure for inferring the utility function of the scope insensitive people who were thinking about saving one or eight children was flawed too, in a different way.
Humans can't really have unbounded utility. The brain structures that represent those preferences have finite size, so they can't intuit unbounded quantities. I believe you care some finite amount for all the rest of humanity, and the total amount you care asymptotically approaches that limit as the number of people involved increases to infinity. The marginal utility to you of the happiness of the trillionth person is approximately zero. Seriously, what does he add to the universe that the previous 999,999,999 didn't already give enough of?
Straw man argument. You're the only one who mentioned paying to kill children.
I'm afraid that backing away from the whole "one child over eight" think but standing by the rest of scope sensitivity doesn't save you from killing. For example, if you value ten million people only twice as much as an a million then you can be persuaded to prefer 20% chance of death for 10 million over certain death for 1 million, which means, on average, condemning 1 million people to death.
Any utility function that does not assign utility to human life in direct proportion to the number of lives at stake is going to kill people in some scenarios.
I go with the 'revealed preference' theory of utility myself. I don't think the human brain actually includes anything that looks like a utility function. Instead, it contains a bunch of pleasure pain drives, a bunch of emotional reactions independent of those drives, and something capable of reflecting on the former two and if necessary overriding them. Put together, under sufficient reflection, these form an agent that may act as if it had a utility function, but there's no little counter in its brain that's actually tracking utility.
Thus, the way to deduce things about my utility function is not to scan my brain, but to examine the choices I make and see what they reveal about my preferences. For example, I think that if faced with a choice between saving n people with certainty and a 99.9999% chance of saving 2n people, I would always pick the latter regardless of n (I may be wrong about this, I have never actually faced such a scenario for very large values of n). This proves mathematically that my utility function is unbounded in lives saved.
The merit of those alternatives depends on how many people total there are. If there are only 10 million people, I'd much rather have 1 million certain deaths than 20% chance of 10 million deaths, since we can repopulate from 8 million but we can't repopulate from 0.
Even if condemning 1 million to death on the average is wrong when all options involve the possible deaths of large numbers of people, deriving positive utility from condemning random children to death when there's no dilemma is an entirely different level of wrong. Utility as a function of lives should flatten out but not slope back down, assuming overpopulation isn't an issue. The analogy isn't valid. Let's give up on the killing children example.
Yes! Agreed completely, in cases where doing the experiment is practical. All our scenarios seem to involve killing large numbers of people, so the experiment is not practical. I don't see any reliable path forward -- maybe we're just stuck with not knowing what people prefer in those situations any time soon.
If 2n is the entire population, in one case we have 0 probability of ending up with 0 people and in the other case we have 0.0001% chance of losing the entire species all at once. So you seem to be more open to mass suicide than I'd like, even when there is no simulation or extortion involved. The other interpretation is that you're introspecting incorrectly, and I hope that's the case.
Someone voted your comment down. I don't know why. I voted your comment up because it's worth talking about, even though I disagree.
Okay, I guess the long term existence of the species does count as quite a significant externality, so in the case where 2n made up the whole species I probably would (I generally assume, unless stated otherwise, that both populations are negligible proportions of the species as a whole.
However, I don't think humanity is a priori valuable, and if humanity now consists of 99.9% simulations being tortured then I think we really are better off dead.
It may be that, in a certain sense, one is more 'wrong' than the other. However, both amount to an intentional choice that more humans die, and I would say that if you value human life, they are equally poor choices.
How can 10 million humans not be 10 times as valuable as 1 million humans. How does the value of a person's life depend on the number of other people in danger?
I'm inclined to say that my intuitions are probably fairly good on these sort of hypothetical scenarios, provided that the implications of my choices are quite distant and do not affect me personally (i.e. I would be more sceptical of my intuitions if I was in one of the groups).
I already answered that. The first few hundred survivors are much more valuable than the rest. Even if survival isn't an issue, the trillionth human adds much less to what I value about humanity than the 100th human does.
I haven't seen any argument for total utility being proportional to total number of people other than bald assertions. Do you have anything better than that?
It's your choice whether you count those simulations as human or not. Be sure to be aware of having the choice, and to take responsibility for the choice you make.
You're human and you're saying that humanity is not a priori valuable? What?
Heavily for most people - due to scope insensitivity. Saving 1 person makes you a hero. Saving a million people does not produce a million times the effect. Thus the size sensitivity.
Heavily for most people - due to scope insensitivity. Saving 1 person makes you a hero. Saving a million people does not produce a million times the effect. Thus the size sensitivity.
Yeah, just realized that I've just written, in a place where anyone who wants can see, that I would be prepared to wipe out humanity. :O
I'm just lucky that nobody really cares enough to search the entire internet for incriminating comments. :)
I'm not the person who downvoted you, but I suspect the reason was that when you said this:
You implied that it's wrong or nonsensical to care about other people's happiness/absence of suffering as a terminal value. We are "allowed" to have whatever terminal values we want, except perhaps contradictory ones.
That's presumably why he said "my."
I don't know what it means for a person to be simulated. I don't know if the simulated people have consciousness. Are we talking about people whose existence feels as real to themselves as it would to us? This is NOT an assumption I ever make about simulations, but should I consider it so for the sake of the argument?
He should or shouldn't exterminate his flesh-and-blood species based on whether his utility function assigns a higher value to a free (and aliive) humanity, than to a trillion of individual sentients being happy.
On my part, I'd choose for a free and alive humanity still. But that's an issue that depends on what terminal values we each have.
Um, I never tried to define sanity. What are you responding to?
Apologies, I did indeed misremember who it was that was talking about "crazy notions", that was indeed Perplexed.
You seem to be collecting some downvotes that should have gone to me. To even things out, I have upvoted three of your comments. Feel free to downvote three of mine.
I fully agree, by the way, on the distinction between the moral relevance of simulated humans, who have no ability to physically influence our world, and the moral relevance of distant people here on earth, who physically influence us daily (though indirectly through a chain of intermediary agents).
Simulated persons do have the ability to influence us informationally, though, even if they are unaware of our existence and don't recognize their own status as simulations. I'm not sure what moral status I would assign to a simulated novelist - particularly if I liked his work.
ETA: To Normal_Anomaly: I do not deny people the right to care about simulations in terms of their own terminal values. I only deny them the right to insist that I care about simulations. But I do claim the right to insist that other people care about Chinese, for reasons similar to those Tim has offered.
But where's the drama in that?
:-)
Relevant to your interests, possibly.
Thanks! Cute story.