= 27d567bc2d48d9f40e9ccc20ea370b15)
Larks comments on Transparency and Accountability - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (141)
Even then the expected lives saved by SIAI is ~10^28.
It's patently obvious that SIAI has an annual income of less than $10^6.
Suppose the marginal dollar is worth 10^3 times less than the average dollar.
Even then a yearly donation of $1 saves an expected10^18 lives.
A yearly donation of £1 to VillageReach saves 100,000,000,000,000,000,000 fewer people.
*Sorry Clippy, but multifoliaterose is damaging you here; SIAI is a lot more amenable to negotiation than anyone else.
A problem with Pascal's Mugging arguments is that once you commit yourself to taking seriously very unlikely events (because they are multiplied by huge potential utilities), if you want to be consistent, you must take into account all potentially relevant unlikely events, not just the ones that point in your desired direction.
To be sure, you can come up with a story in which SIAI with probability epsilon makes a key positive difference, for bignum expected lives saved. But by the same token you can come up with stories in which SIAI with probability epsilon makes a key negative difference (e.g. by convincing people to abandon fruitful lines of research for fruitless ones), for bignum expected lives lost. Similarly, you can come up with stories in which even a small amount of resources spent elsewhere, with probability epsilon makes a key positive difference (e.g. a child saved from death by potentially curable disease, may grow up to make a critical scientific breakthrough or play a role in preserving world peace), for bignum expected lives saved.
Intuition would have us reject Pascal's Mugging, but when you think it through in full detail, the logical conclusion is that we should... reject Pascal's Mugging. It does actually reduce to normality.
Related:
How Much it Matters to Know What Matters: A Back of the Envelope Calculation
This is just Pascal's mugging again.
If you get to make up the numbers then you can make things turn out however you like.
Not if you deliberately pick numbers orders of magnitude more unfavourable to your point of view than you think is actually the case.
But you didn't do that. A calculation involves both quantitative and qualitative/structural assumptions, and you slanted all of your qualitative choices toward making your point.
You used a discount rate of 0. (That is, a hypothetical life a hundred generations from now deserves exactly as much of my consideration as people alive today). That totally discredits your calculation.
You used a definition of 'saving a life' which suggests that we ought to pay women to get pregnant, have children, and then kill the infant so they are free to get pregnant again. Count that as one life 'saved'.
You didn't provide estimates of the opportunity cost in 'human lives' of diverting resources toward colonizing the universe. Seems to me those costs could be enormous - in lives at the time of diversion, not in distant-future lives.
Appart from the issue JGWeissman brings up, even if you supposed that the lives saved all occurred 300 years in the future, a reasonable discount rate would still only give you a couple of orders of magnitude.
For example, 1.05^300 = 2.3 * 10^6
Which is nowhere near enough.
Edit: there are discount rates that would give you the result you want, but it still seems pritty plausible that, assuming Astronomical Waste, SIAI isn't a bad bet.
Sounds about right to me.
Huh? You think 6 is "a couple"? I wish I had your sex life!
But 5% per annum is far too high. It discounts the next generation to only a quarter of the current generation. Way too steep.
Double huh? And huh? some more. You wrote:
You imagine (conservatively) that there are a potential 10^18 lives to save 300 years into the future? Boy, I really wish I had your sex life.
If people 300 years from now are whole brain emulations or AIs, then they could reproduce like software with high population densities.
Alternatively, if the human-size brains were all sucked into the matrix long ago, there may well be about 1 person per-planet.
What makes a life at one time worth more than a life at a different time?
In a sense, these are flip answers, because I am not really a utilitarian to begin with. And my rejection of utilitarianism starts by asking how it is possible to sum up utilities for different people. It is adding apples and oranges. There is no natural exchange rate. Utilities are like subjective probabilities of different people - it might make sense to compute a weighted average, but how do you justify your weighting scheme?
I suspect that discussing this topic carefully would take too much of my time from other responsibilities, but I hope this sketch has at least given you some things to think about.
Utilities for different people don't come into it. The question is, how much to you now is a contemporaneous person worth, versus someone in a hundred generations? (Or did you mean utilities of different people?)
Utilities of different people, yes. He's complaining that they don't add up.
You have me confused now. I like apples. Joe likes oranges. Mary wishes to maximize total utility. Should she plant an orange tree or an apple tree?
I agree with Kant (perhaps for the only time). Asking how much Joe is worth to Mary is verbotten.
She should consider the world as it would be if she planted the apple tree, and the world as it would be if she planted the orange tree, and see which one is better as she measures value. (Another option is to plant neither, of course.)
The idea isn't to add up and maximize everyone's utility. I agree with you that that makes no sense. The point is, when an agent makes a decision, that agent has to evaluate alternatives, and those alternatives are going to be weighed according to how they score under the agent's utility function. But utility isn't just selfish profit. I can value that there be happiness even if I don't ever get to know about it.
We don't always have the luxury of not choosing. What should Mary do in a trolley problem where she has to direct a train at either you or Joe (or else you both die)? That said . . .
"Worth" needs to be understood in an expansive sense. Kant is probably right that Mary shouldn't think, "Now, from whom can I extract more profit for my selfish ends? That's the one I'll save." The things she ought to consider are probably the ones we're accustomed to thinking of as "selfless". But she can't evade making a decision.
Ok, I think we can agree to agree. Revealed preference doesn't prevent me from incorporating utilitarianish snippets of other peoples utility judgments into my own preferences. I am allowed to be benevolent. But simple math and logic prevent me from doing it all-out, the way that Bentham suggested.
Now, which one of us has to tell Eliezer?
No, what is forbidden is to ask how much Joe is worth in an absolute sense, independent of an agent like Mary.
Utility is not a fundamental property of the world, it is perceived by agents with preferences.
This is rapidly becoming surreal.
Forbidden is not a fundamental property of the world, it is imposed by theorists with agendas.
Samuelson, Wald, von Neumann, Savage, and the other founders of "revealed preference" forbid us to ask how much Joe (or anything else) is worth, independent of an agent with preferences, such as Mary.
Emmanuel Kant, and anyone else who takes "The categorical imperative" at all seriously, forbids us to ask what Joe is worth to Mary, though we may ask what Joe's cat Maru is worth to Mary.
I knew I shouldn't have gotten involved in this thread.
For 2, perhaps consider:
http://lesswrong.com/lw/n2/against_discount_rates/
Considered. Not convinced. If that was intended as an argument, then EY was having a very bad day.
He is welcome to his opinion but he is not welcome to substitute his for mine.
The ending was particularly bizarre. It sounded like he was saying that treasury bills don't pay enough interest to make up for the risk that the US may not be here 300 years from now. But we should, for example, consider the projected enjoyment of people we imagine visiting our nature preserves 500 years from now, as if their enjoyment were as important as our own, not discounting at all for the risk that they may not even exist.
Time travel, the past "still existing" - and utilitariainism? I don't buy any of that either - but in the context of artificial intelligence, I do agree that building discounting functions into the agent's ultimate values looks like bad news.
Discounting functions arise because agents don't know about the future - and can't predict or control it very well. However, the extent to which they can't predict or control it is a function of the circumstances and their own capabilities. If you wire temporal discounting into the ultimate preferences of super-Deep Blue - then it can't ever self-improve to push its prediction horizon further out as it gets more computing power! You are unnecessarily building limitations into it. Better to have no temporal discounting wired in - and let the machine itself figure out to what extent it can predict and control the future - and so figure out the relative value of the present.
Eliezer doesn't disagree: as he says more than once, he's talking about pure preferences, intrinsic values. Other risks do need to be incorporated, but it seems better to do so directly, rather than through a discounting heuristic. Larks seems to implicitly be doing this with his P(AGI) = 10^-9.
Well, in that case, your math is all wrong!
The SIAI is actually having a negative effect, by distracting conscientious researchers with all kinds of ridiculous and irrelevant moral issues - which results in the unscrupulous Korean FKU corporation developing machine intelligence first - when otherwise they would have been beaten to it. Their bot promises humans a cure for cancer - but then it gets powerful, eats the internet, and goes on to do all manner of dubious things.
If only you had had the sense to give to VillageReach!
With such low numbers the incidental effects of VillageReach on existential risk (via poverty, population, favoring the efficient philanthropy culture, etc) would dominate. If reasonably favorable, they would mean VillageReach would be a better choice. However, the numbers in the comment are incredibly low.
We need to be careful with the terminology here. "Saving expected lives" and "expecting to save lives" are pretty different activities.
Thanks & corrected.
Wow. Do you really think this sort of agrument can turn people to SIAI, rather than against any cause that uses tiny probabilities of vast utilities to justify itself?
Seconded, tentatively. I'm afraid that all arguments of this form (for x-risk reduction, Pascal's Wager, etc.) can't avoid being rejected out of hand by most people, due to their insatiability: buying even a little bit into such an argument seems to compel a person to spend arbitrary amounts of resources on the improbable thing, and open them to harsh criticism for holding back even a little. That said, a few sincere consequentialists actually will react positively to such arguments, so maybe making them on LW is worthwhile on balance.
Multifoliaterose said his result held even for donors who took Astronomical Waste seriously. This seems unlikely to be the case.
Edit: I didn't vote you down, but what? SIAI is an Existential Risk charity: the point is to save the entire human race, at low probability. Of course the expected value is going to be calculated by multiplying a tiny probability by an enormous value!
This isn't true for all existential risks. For example, fears about nuclear war or global warming don't rely on such tiny probabilities. But discussions of many other risks remind me of this xkcd.
The usual consequentialist case for charity to reduce risks of nuclear war or catastrophic global warming feedbacks does rely on tiny probabilities of donations making a difference. Likwise for voting and for many kinds of scientific and medical research charity.
Edit: not as tiny as in Lark's comment, although the numbers there are incredibly low.
If the Pascal's mugging issue is with exerting a small force on a big thing and therefore having a small probability of succeeding, I don't think that's even a coherent objection; in a chaotic universe, anything you do may save or destroy individual people and human civilization by redrawing from the distribution, and shifting the distribution of the number of saved lives up by one through e.g. aid or buying someone cryo doesn't seem fundamentally different from e.g. shifting the distribution of election outcomes. This is more clearly true if you believe in a multiverse such as MWI, but I don't think it requires that.
ETA: if your distribution on people killed is uniform for say 6 through 10, then definitely saving one person and having a 1 in 5 chance of turning a 10-death situation into a 5-death situation are the same thing except for maybe the identity of the victims, right?
That's a very different value of "tiny" than the one in Larks's comment.