Meetup : Toronto THINK
Discussion article for the meetup : Toronto THINK
In the Hart House Reading Room (the room across from the reception desk with the purple walls)
Sustainability and the environment are hot topics - global warming, resource depletion, biodiversity loss, overpopulation, water resources, pollution, deforestation. But as individuals, what should we be doing about it? Should we be trying to minimize our own environmental impact, or encouraging our friends to do so? Or should we be lobbying for change at the political level? Just how much do things need to change? Is it already too late? And, importantly, which issues should be getting most of our attention and which do we think are distractions?
This will be the focus of next week's discussion.
In addition to this, we're starting a project to save a life! One of the best ways we know to go about this comes to our attention via the charity evaluator GiveWell. If their research is to be believed, the Against Malaria Foundation can save a life for around $2000. But we need to get the money somehow, and we need to decide whether we trust GiveWell's methodology and whether there are better options available. All things to discuss at the next meetup!
Note: THINK is not directly LW-affiliated but I've been told to post our meetups here anyhow :-)
Discussion article for the meetup : Toronto THINK
Meetup : Toronto THINK
Discussion article for the meetup : Toronto THINK
In the Hart House Reading Room (the room across from the reception desk with the purple walls)
How To Save A Life
It's time to discuss one of the most basic ways to make the world better - saving somebody's life. How would you go about that? How much of an inconvenience do you think it would be? In this meeting we'll discuss these issues. We might even come up with a plan we can put into practice.
Note: THINK is not directly LW-affiliated but I've been told to post our meetups here anyhow :-)
Discussion article for the meetup : Toronto THINK
Notes from Online Optimal Philanthropy Meetup: 12-10-09
Here are my notes from the Optimal Philanthropy online meeting. Things in square brackets are my later additions and corrections. Let me know if there are factual errors or if I've incorrectly captured the drift of what you were saying.
Nick:
- existential risk argument goes as follows:
- small chance that people will be around for a really long time
- if so, increasing probability of that happening by a small amount then really good
- therefore, you should focus on this rather than things that have short-run impacts
- focus only on things that reduce xrisk
- Similar property: make things a little bit better for a really long time. Might be easier to do that.
- name for these: trajectory change. Different from just economic growth – changes default trajectory along which things go
- interesting for same reason as xrisk reduction but people haven’t talked much about them.
[sorry, didn’t note who said this]: education as an example. Particular kind of education?
Nick:
- not really thought through this idea yet. [This idea is at the same stage] as if you’d just stumbled across xrisk concept
- another example is just changing people’s values for a better
- ordinary – e.g. campaigning for particular political party
Scott:
- calls this “static” [I assume this means as in “noise” rather than as in “stationary”]
- I’m doing something – tends to break down after a month or so, very difficult to predict long term results
- should focus on things that can be predicted, build better models
Nick:
- things that look short run could have long run impacts
- e.g. thinking of buying bednets for people as having short-term impact – has short run impact today, no impact in 100 years time
- but if you think about all downstream consequences, actually making small impact
Scott agrees.
- both good and bad long-term consequences included in static
- it’s a wash
Jeff:
- what makes you think bednets have good long term consequences?
- currently thinks of bednet as short term good, long term wash
- benefits of having more people, having more healthy kids?
- really hard to predict long term value
Nick:
- Four classes of thing
- things that reduce xrisk
- things that speed up progress
- things that make trajectory change
- only short run effect
- bednet not a trajectory change.
- buying bednet increases economic growth – long term effects
- economic growth has been compounding for a long time.
- Plausible story for why this happens – people get richer, specialization, help each other out. Overall a positive process, and a process that ripples far into the future.
Jeff:
- really hard to get good predictions. Maybe you could figure it out
- even evaluating short-term effects of individual charity now is hard.
- long term effects incredibly difficult
Nick:
- when comparing within categories, makes sense to ignore these thing.
- e.g. two things that contribute to economic growth – just ask which has better predictable impacts.
- but if you’re comparing something that reduces xrisk with something that increases economic growth
Jeff:
- funding education versus funding health.
- wouldn’t surprise him if someone got handle on their long term effects and they were very different
Nick:
- expects one of them to contribute to trajectory change more
- hard to believe one of them has a very different effect on long term economic growth
- e.g. education appeared on a par with health intervention, but expecting 1000x more economic growth seems wild.
Jeff:
- [if optimizing for] economic growth – would do it very differently from doing health interventions
Nick:
- can’t use economic growth argument to promote AMF: "AMF is best charity to
- more plausible candidate: political or meta-research (some GiveWell thing)
At this point everyone on the call introduces themselves.
- Giles Edkins: Toronto LW, THINK
- Jeff Kaufman: married to Julia Wise. Both into earning to give stuff
- Nick Beckstead: PHD in Philosophy at Rutgers. Population ethics and xrisks. Involved in Centre for Effective Altruism.
- Nisan Stiennon: Berkeley. PHD in Math. Teaches for CFAR. Confused about whether to focus on xrisk or other kinds of philanthropy e.g. AMF
- Peter Hurford: Ohio. Political Science & Psych. Joined GWWC – giving 10% of meager college income. New to smart giving stuff, on smart giving subreddit, on LW.
- Raymond Arnold: does a lot of work for THINK. LW for past two years. General boat of xrisk & GiveWell stuff, only things he has a lot of info about
- Scott Dickey: LWer, gives 25% of income to charity, trying to make that go as far as he can – SingInst, GiveWell. Following, reading them, watching where things are going
- Boris Yakubchik: president of GWWC Rutgers. Giving 50% of income, high school math teacher. Don’t know what to expect from this hangout – needs to run off!
Ray:
- assumes everyone has read GW, Nick Bostrom’s xrisk stuff
- anyone read anything else?
Nick:
- a lot of my dissertation is not empirical, instead it’s moral philosophy
- has been thinking on the side on comparison between xrisk and other kinds of things, talking to people
- feeling is there’s not huge amount of writing
- e.g. economics of climate change – most similar to xrisk. Not a whole lot out there that’s helpful
Ray:
- earlier this summer (interrupted by other things), wanted breadth-first look at potentially promising high impact stuff
- 3 books on global poverty, each with a different thesis
- The End of Poverty – Jeffrey Sachs
- White man’s burden – William Easterly
- The Bottom Billion – [Paul Collier]
- would be good for people in our community to start compiling summaries of this information
Giles:
- GW prioritizes things that are easier to understand – targets beginners
Ray:
- And where there’s good information
Jeff:
- GW are reasonably honest in not just choosing charities that are accessible, but that the people involved are convinced about the best ones – has talked to Holden on Skype 8 months ago.
Ray:
- Only recently with GW Labs have they made an effort to look at harder questions, not just looking at charities with easy-to-obtain information
Giles:
- Any info on how GW Labs is going?
Ray:
- 100k to Cochrane. Not a whole lot of money! Still question on what else is out there
Peter:
- Thought they were going to move away from GW Labs, were going to pursue this as main funding strategy
Scott:
- Thought they were going to integrate things into one effort
Peter:
- target big funders
- away from health related issues
Scott:
- [Should we] give to GiveWell itself?
Ray:
- [If we give to GiveWell], a small amount to funding goes to themselves, the rest goes to their favorite charity
Peter:
- can give to Clear Fund – fund for GW activities.
- But they strongly discourage that – want to give to their top charity instead.
Giles:
- can we talk to GW and find out why?
Jeff:
- if you give to charity, tell them it’s because of GW
Ray:
- has anyone talked to non-LWers about optimal philanthropy?
Jeff:
- talked to college friends, not LWers. Some people think it makes sense – they are the people most similar to LWers, computer programmers and other engineers. Job makes them a bit more money than they know what to do with. “If I started giving away more than 20% of my money it wouldn’t hurt.” Helps people to think about it dispassionately.
- Other people not inclined to this approach – doesn’t like quantifying. Shouldn’t be prioritising one thing over another
Scott:
- talk to people at work, highly diverse set of backgrounds. Vast majority doesn’t like analytic side – they call me “cold”. Work fundraising drive – literally had picture of crying puppy. Very emotional, very community oriented, didn’t like analysis. Could be because I’m a bad arguer.
Jeff:
- tries really hard, really hard to have discussions with people that people aren’t giving to best charity. People used to get mad at him. Now usually agrees to disagree but not losing any more friends
Ray:
- requested charity Christmas gift – think about who you’re giving to. Conversation over Christmas dinner – worst idea ever! Grandmother most likely to donate, donated to local causes. She prioritizes giving locally not because she believes its the most effective but she prioritises her own community [did I get this right?]
- nowadays asks what being a good person means to someone before having that conversation
- has been having conversation at work not as “you should be doing it too” but “here’s this thing I’m doing”
- has laid groundwork.
- one person has said she now gives to more global causes [is this right?] as a result of what I was saying
- long-term non-confrontational strategy
Scott:
- another thing GW is doing right is targeting high-wealth individuals
- spent a lot of time talking to my Dad about it. After 10 conversations he says “fine, I’ll give to what you think are more effective causes” – gave $100, and that was it forever.
Jeff:
- convincing someone to give $10000 a year is not something to be sneezed at. Need to accept both the “give effectively” and “give substantially” ideas.
Jeff:
- not many people [in effective altruism movement] older than I am. (late 30’s or higher)
Scott:
- Bill Gates and Warren Buffett started own movement. Get rich people to give 50% of resources
- Jaan Tallin – level 2 – looking at “malaria” instead of “human disease”
Jeff:
- B&M Gates Foundation does a lot of stuff with malaria because it’s effective – same reason as GiveWell.
Scott: agrees
- goes back to Nick’s thing of smaller things making larger impacts over time
- not trying to change human condition
- what inspired Scott: “what can you do today that people can remember your name 50000 years for now”
Ray:
- if goal is to be remembered, won’t get there by donating. Founding agency or inventing something.
Scott:
- Or Hitler, but would rather not be on that list.
Ray:
- new ways of thinking – Darwin, Galileo.
- optimizing giving is a different question
- even “change the way governments work” – won’t be remembered for donating to that. But that’s OK.
- agrees with sentiment though
Jeff:
- certainly an inspiring sentiment
Ray:
- what is the thing you’d want to remembered in 50000 years for, even if your name isn’t remembered?
Jeff:
- some really important things don’t get remembered, e.g. eradicating Polio and Smallpox. Huge amounts of work, people don’t think about it because it’s no longer a problem.
Scott:
- amend it to “should” be remembered for
Ray:
- something I’ve been trying to do is put together a list of inspiring things that humans have done that we should care about.
- evidence that things are getting better
- ways of dealing with stress – global poverty is really depressing
- “we should do more stuff like that”
- polio & smallpox on list
Scott:
- green revolution
- one guy, saved or created billions of lives
Nick:
- book: “millions saved” [Ruth Levine?], success stories in philanthropy in particular global health
- Steven Pinker: “better angels of our nature, chapter 1”, things that went surprisingly well in last few decades
- “Scientists greater than Einstein” [Billy Woodward?]
Ray:
- 80000hours blog
- have read it occasionally
- first few times, changed how I looked at that side of things
- since then, haven’t found a whole lot of content
Scott:
- stopped reading 80k
- they deferred their choosing of charities to GW
- [I had] already read up on income maximization
Jeff:
- am subscribed to 80k blog
- haven’t read things that are wholly new ideas that I’m excited to read
- mostly been summaries of things
- remaining questions are really hard. People won’t make good progress sitting down for an afternoon reading [writing?] a blog post
Scott:
- waiting for Nick Bostrom to come up with magnum opus?
Jeff:
- happy with what I’m doing so far
- important not to get attached to really high leverage factors where it’s only worth it if impact is too huge to think about
- good to realize how much good you can do with a GWWC-style 10% given to GW charities
- and try and do better than that!
Ray:
- Jeff: you and Julia are highest% giving couple that I know of. On the giving a lot side…
Jeff:
- keeping a donations page, very numbersy
- Bolder Giving talks about us as if it’s still 2009 and we have changes to make, titled Julia Wise and should be both of us…
Ray:
- question is “are they living in Manhattan”?
Jeff:
- rent in Boston is still high
- studio apartment costing $1000 a month
- even if we’d be spending twice as much, it wouldn’t have had a huge effect
- actually, yes it would have a huge effect! [but could still give away large percentage]
Ray:
- and do they have kids?
Jeff:
- yes, planning to
- yearly expenses will change by $10000 for each kid we have [note I may have written down Jeff’s numbers wrong]
- living at parents house, pay them rent but it’s low
- own house with kids: $40000 on ourselves, giving at least that much away, still pretty good
- looking forward to say “yes, you can have kids and a house and still give away lots!”
Giles:
- “is that your true reason for rejection”
- people still might manage to find a reason why you’re weird and not want to emulate you?
- have house & kids because you want them, not for signaling!
Jeff:
- yes, of course!
- more normal and conventional our life seems, convincing in a non-logical way
- “oh, I could live a life like that. Not so different from the life I was thinking of living”.
Nick:
- a few people had a question about how to compare xrisk reduction and GW charities
- are there specific aspects of that that people want to know about?
Giles:
- how do we know xrisk orgs are having any impact at all? How do we know they’re not making things worse?
Scott:
- Yudkowksy’s scale of ambition. Went into hacker news post, restructured entire conversation where highest you could go was “bigger than apple”, he put that at 2. His 10 was “you can hack the computer universe is running on”
Ray:
- expected value calculation – don’t need to know they [xrisk orgs] are doing something meaningful, 1% chance is good enough. But how can we even be sure of that?
- without some calculation, running on intuition.
- intuitively sounds plausible that donating to xrisk org is higher impact than giving to AMF
- whole point of GW is that our intuitions aren’t necessarily reliable
- what are the numbers for xrisk?
- one thing that’s bothering him – people say xrisk, only think of SingInst, FHI
Scott:
- Methuselah Foundation
- Lifeboat Foundation (even they sound amateurish)
Ray:
- SingInst is most reputable of the various xrisk orgs [what about FHI?], still has a lot of reasons to be skeptical of SingInst
- changing a bit since Luke Muelhauser took the reins, moving in more reliable direction
[These didn't all come up in the discussion, but I'll give lukeprog's list of x-risk orgs: FHI, FutureTech, the Singularity Institute, Leverage Research, the Global Catastrophic Risk Institute, CSER]
Scott:
- Growing pains
Ray:
- OK, I can see SingInst is improving. Not at point I’d be willing to donate to them yet
- set up criteria: if they meet these criteria, I’d consider them reliable
Giles:
Ray:
- part of what influenced me
Nick:
- do you wish there was more like the Holden post?
Scott:
- Holden should target any other organization in the field [i.e. should address other xrisk orgs besides SingInst]
Jeff:
- Holden’s post on Singinst was mostly organisation critique. Looking at what they’re doing, things they’re doing that seem to be working and not.
- even if they passed all of those things, I would still be unsure it would make more sense [to donate] than an easily measurable intervention [such as AMF]
- SingInst still doing incredibly hard to evaluate work
Ray:
- asteroid impact – already solved/diminishing returns point
- AI risk
- nuclear proliferation
- bioterrorism
- CDC
[question came up on what other xrisk mitigation efforts might there be that we don’t know about, in particular AI related]
Scott:
- Google, Microsoft Research
Jeff:
- Google mostly public with google.org stuff
- what secret but massively positive stuff would there be?
Scott:
- Google cars, save 30000 lives a year, not too big. Bigger if roll out to world
- at that point, Google is AI company and should be focusing on AI safety
Jeff:
- people in Google mostly think that kind of thing is silly?
- very far from machine learning systems that are scary
- so hard to get them to do anything at all intelligent
- hard for people to think about possibility of recursively self-improving anything.
- not my impression that they are excited about AI safety.
Scott:
- not saying that they are, but that they should be
- keep a lot of it locked down
- we don’t know what’s happening in China, Korea, Japan had a push for AI recently
Jeff:
- wouldn’t be surprised if no-one at Google knows more about AI safety than what you can read on LW
Ray:
- one thing that’s potentially high impact is donating to scientific research
- there’s a thing called Petridish – Kickstarter for research projects
- not optimized for “these are the projects that will help the world the most”
- wonder if there’s a way we can push this in that direction?
Jeff:
- amazing how animal-focused they are
- specific to biology, or is that what people like to fund?
Giles:
- find people good at communicating effective altruism message and put them in touch with orgs [such as Petridish] that need it?
Scott:
- finding marketers
Ray:
- some of that is in progress at Leverage. At pre-planning stage
Scott:
- campaign Red or Pink, generate millions of dollars by having campaigns. Doesn’t seem to be having much impact, but can we co-opt or create something like that?
Ray:
- am definitely looking for info on xrisk in a more concise form
- “I don’t have a degree in computer science or AI research, wouldn’t be able to analyze at Eliezer level”
- so layman can feel like they’re making an informed decision, has important information but not too much information
Babyeater's dilemma
Imagine it's the future, and everything has gone according to plan. Humanity has worked out its own utility function, f0, and has worked out a strategy S0 to optimize it.
Humanity has also run a large number of simulations of how alien worlds evolve. It has determined that of those civilizations which reach the same level of advancement - that know their own utility function and have a strategy for optimizing it - there is an equal probability that they will end up with each of 10 possible utility functions. Call these f0...f9.
(Of course, these simulations are coarse-grained enough to satisfy the nonperson predicate).
Humanity has also worked out the optimal strategy S0...S9 for each utility function. But they just happen to score poorly on all of the others:
fi(Si) = 10
fi(Sj) = 1 for i != j
In addition, there is a compromise strategy C:
fi(C) = 3 for all i.
The utility functions, f0 through f9, satisfy certain properties:
They are altruistic, in the sense that they care just as much about far-away aliens that they can't even see as they do about members of their own species.
They are additive: if one planet implements Sj and another implements Sk, then:
fi(Sj on one planet and Sk on the other) = fi(Sj) + fi(Sk).
(This is just to make things easier - the problem I'm describing will still apply in cases where this rule doesn't hold).
They are non-negotiable. They won't "change" if that civilization encounters aliens with a different utility function. So if two of these civilisations were to meet, we would expect it to be like the humans and the babyeaters: the stronger would attempt to conquer the weaker and impose their own values.
In addition, humanity has worked out that it's very likely that a lot of alien worlds exist, i.e. aliens are really really real. They are just too far away to see or exist in other Everett branches.
So given these not entirely ridiculous assumptions, it seems that we have a multiplayer prisoner's dilemma even though none of the players has any causal influence on any other. If the universe contains 10 worlds, and each chooses its own best strategy, then each expects to score 19. If they all choose the compromise strategy then each expects to score 30.
Anyone else worried by this result, or have I made a mistake?
Evolution, bias and global risk
Sometimes we make a decision in a way which is different to how we think we should make a decision. When this happens, we call it a bias.
When put this way, the first thing that springs to mind is that different people might disagree on whether something is actually a bias. Take the bystander effect. If you're of the opinion that other people are way less important than yourself, then the ability to calmly stand around not doing anything while someone else is in danger would be seen as a good thing. You'd instead be confused by the non-bystander effect, whereby people (when separated from the crowd) irrationally put themselves in danger in order to help complete strangers.
The second thing that springs to mind is that the bias may exist for an evolutionary reason, and not just be due to bad brain architecture. Remember that evolution doesn't always produce the behavior that makes the most intuitive sense. Creatures, including presumably humans, tend to act in a way as to maximize their reproductive success; they don't act in the way that necessarily makes the most intuitive sense.
The statement that humans act in a fitness-maximizing way is controversial. Firstly, we are adapted to our ancestral environment, not our current one. It seems very likely that we're not well adapted to the ready availability of high-calorie food, for example. But this argument doesn't apply to everything. A lot of the biases appear to describe situations which would exist in both the ancestral and modern worlds.
A second argument is that a lot of our behavior is governed by memes these days, not genes. It's certain that the memes that survive are the ones which best reproduce themselves; it's also pretty plausible that exposure to memes can tip us from one fitness-maximizing behavioral strategy to another. But memes forcing us to adopt a highly suboptimal strategy? I'm sceptical. It seems like there would be strong selection pressure against it; to pass the memes on but not let them affect our behavior significantly. Memes existed in our ancestral environments too.
And remember that just because you're behaving in a way that maximizes your expected reproductive fitness, there's no reason to expect you to be consciously aware of this fact.
So let's pretend, for the sake of simplicity, that we're all acting to maximize our expected reproductive success (and all the things that we know lead to it, such as status and signalling and stuff). Which of the biases might be explained away?
The bystander effect
Eliezer points out:
We could be cynical and suggest that people are mostly interested in not being blamed for not helping, rather than having any positive desire to help - that they mainly wish to escape antiheroism and possible retribution.
He lists two problems with this hypothesis. Firstly, that the experimental setup appeared to present a selfish threat to the subjects. This I have no convincing answer to. Perhaps people really are just stupid when it comes to fires, not recognising the risk to themselves, or perhaps this is a gaping hole in my theory.
The other criticism is more interesting. Telling people about the bystander effect makes it less likely to happen? Well, under this hypothesis, of course it would. The key to not being blamed is to formulate a plausible explanation; the explanation "I didn't do anything because no-one else did either" suddenly sounds a lot less plausible when you know about the bystander effect. (And if you know about it, the person you're explaining it to is more likely to as well. We share memes with our friends).
The affect heuristic
This one seems quite complicated and subtle, and I think there may be more than one effect going on here. But one class of positive-affect bias can be essentially described as: phrasing an identical decision in more positive language makes people more likely to choose it. The example given is "saving 150 lives" versus "saving 98% of 150 lives". (OK these aren't quite identical decisions, but the difference in opinion is more than 2% and goes in the wrong direction). Apparently putting in the word 98% makes it sound more positive to most people.
This also seems to make sense if we view it as trying to make a justifiable decision, rather than a correct one. Remember, the 150(ish) lives we're saving aren't our own; there's no selective pressure to make the correct decision, just one that won't land us in trouble.
The key here is that justifying decisions is hard, especially when we might be faced with an opponent more skilled in rhetoric than ourselves. So we are eager for additional rhetoric to be supplied which will help us justify the decision we want to make. If I had to justify saving 150 lives (at some cost), it would honestly never have occurred to me to phrase it as "98% of 153 lives". Even if it had, I'd feel like I was being sneaky and manipulative, and I might accidentally reveal that. But to have the sneaky rhetoric supplied to me by an outside authority, that makes it a lot easier.
This implies a prediction: when asked to justify their decision, people who have succumbed to positive-affect bias will repeat the postive-affective language they have been supplied, possibly verbatim. I'm sure you've met people who quote talking points verbatim from their favorite political TV show; you might assume the TV is doing their thinking for them. I would argue instead that it's doing their justification for them.
There is a class of people, who I will call non-pushers, who:
- would flick a switch if it would cause a train to run over (and kill) one person instead of five, yet
- would not push a fat man in front of that train (killing him) if it could save the five lives
So what's going on here? Our feeling of shouldness is presumably how social pressure feels from the inside. What we consider right is (unless we've trained ourselves otherwise) likely to be what will get us into the least trouble. So why do non-pushers get into less trouble than pushers, if pushers are better at saving lives?
It seems pretty obvious to me. The pushers might be more altruistic in some vague sense, but they're not the sort of person you'd want to be around. Stand too close to them on a bridge and they might push you off. Better to steer clear. (The people who are tied to the tracks presumably prefer pushers, but they don't get any choice in the matter). This might be what we mean by near and far in this context.
Another way of putting it is that if you start valuing all lives equally, and not put those closest to you first, then you might start defecting in games of reciprocal altruism. Utilitarians appear cold and unfriendly because they're less worried about you and more worried about what's going on in some distant, impoverished nation. They will start to lose the reproductive benefits of reciprocal altruism and socialising.
Global risk
In Cognitive Biases Potentially Affecting Judgment of Global Risks, Eliezer lists a number of biases which could be responsible for people's underestimation of global risks. There seem to be a lot of them. But I think that from an evolutionary perspective, they can all be wrapped up into one.
Group Selection doesn't work. Evolution rewards actions which profit the individual (and its kin) relative to others. Something which benefits the entire group is nice and all that, but it'll increase the frequency of the competitors of your genes as much as it will your own.
It would be all to easy to say that we cannot instinctively understand existential risk because our ancestors have, by definition, never experienced anything like it. But I think that's an over-simplification. Some of our ancestors probably have survived the collapse of societies, but they didn't do it by preventing the society from collapsing. They did it by individually surviving the collapse or by running away.
But if a brave ancestor had saved a society from collapse, wouldn't he (or to some extent, she) become an instant hero with all the reproductive advantage that affords? That would certainly be nice, but I'm not sure the evidence backs it up. Stanislav Petrov was given the cold shoulder. Leading climate scientists are given a rough time, especially when they try and see their beliefs turned into meaningful action. Even Winston Churchill became unpopular after he helped save democratic civilization.
I don't know what the evolutionary reason for hero-indifference would be, but if it's real then it pretty much puts the nail in the coffin for civilization-saving as a reproductive strategy. And that means there's no evolutionary reason to take global risks seriously, or to act on our concerns if we do.
And if we make most of our decisions on instinct - on what feels right - then that's pretty scary.
People who want to save the world
atucker wants to save the world.
ciphergoth wants to save the world.
Dorikka wants to save the world.
Eliezer_Yudkowsky wants to save the world.
I want to save the world.
Kaj_Sotala wants to save the world.
lincolnquirk wants to save the world.
Louie wants to save the world.
paulfchristiano wants to save the world.
Psy-Kosh wants to save the world.
Clearly the list I've given is incomplete. I imagine most members of the Singularity Institute belong here; otherwise their motives are pretty baffling. But equally clearly, the list will not include everyone.
What's my point? My point is that these people should be cooperating. But we can't cooperate unless we know who we are. If you feel your name belongs on this list then add a top-level comment to this thread, and feel free to add any information about what this means to you personally or what plans you have. Or it's enough just to say, "I want to save the world".
This time, no-one's signing up for anything. I'm just doing this to let you know that you're not alone. But maybe some of us can find somewhere to talk that's a little quieter.
[Altruist Support] Fix the SIAI's website (EDIT: or not. I'll do it)
EDIT: This post no longer reflects my current attitude. I'm now signed up as a volunteer for SIAI and will help them with the website and/or whatever else needs doing. Add a comment or contact me if you're curious as to what my attitude was or why it changed.
What I've learnt: People want something more specific
What I've also learnt: Not to commit to donating money to an organization without carefully reading their website first
Imagine you are a prospective SIAI donor. You've learnt about AI and its risks, about how hardly anyone takes these risks seriously, about how people are fundamentally not mentally equipped to handle issues of existential risk in a sane way. You've looked around and seen that the SIAI is the only (or one of just a few) organizations that appear to realise this and want to do anything about it.
So you go to their website. What are you looking for? You're looking for a reason not to give money to them.
The Singularity Institute exists to carry out the mission of the Singularity-aware – to accelerate the arrival of the Singularity in order to hasten its human benefits; ...
This seems a somewhat gung-ho attitude which is not consistent with the message on the rest of the site. And this isn't just my misreading or quoting out of context - apparently that page is very out of date and no longer represents the worldview of the more mature, grown up SIAI.
But people reading the site don't know that. And remember, they're looking for reasons not to give - for reasons to retreat back to their comfort zone where everything's basically OK and the SIAI are just a bunch of weirdos.
The fact that an organization dedicated to shaping the future of humanity can't keep their website up to date would seem to be one of those reasons.
So, if you really believe the SIAI to be the most effective charity right now, you should help them by offering to fix their website for them - in order to help attract more donors.
Some possible objections and counter-objections:
1. If Giles thinks fixing the SIAI's website is so important, he'd already be doing it himself.
Essentially this boils down to the fact that you probably trust the SIAI a lot more than I do. So for me the community-building effort is the higher priority.
2. If the website was so important, the SIAI would already have fixed it. Better just to give them money and they'll spend it on fixing the website when it's optimal to do so.
This assumes that the SIAI behaves in a perfectly rational way. It also ignores the fact that people are going to look at their accounts and try to find evidence that they are actually engaging in saving-the-world type activities. If all they do is "fix our own website and make ourselves look good" then no-one's going to take them seriously. By donating your time to improve their website, you keep that activity off the balance sheet.
3. There are so many more important factors keeping people away from donating to the SIAI. Surely better to address those first?
Maybe - but they need to be fixed one at a time. And I believe the website to be a single point of failure - even people who are otherwise really keen might be put off by a single strange-sounding sentence appearing on the website.
Conclusion:
I don't think the website needs a big overhaul or a massive amount of new information. It just needs a little thought as to people's questions and concerns. Other than the page I mentioned, possible concerns might be:
- All of the issues that arose in the GiveWell interview
- A recognition of the non-strawman criticisms of the SIAI and how they are being addressed
- An answer to the only-game-in-town question: if we recognise that the SIAI is the only organization seriously addressing these issues but aren't sure of its effectiveness, are we better off giving now or waiting for a more effective organization to come along?
[Altruist Support] The Plan
Here's my plan.
I intend to build a community of aspiring rational leaders. I see three components to this:
A. Becoming more rational

I see this as being about bringing together the spheres of self-identity, rationality and the human. The human is your physical body and brain; it is the human which actually does things, and if the human isn't on board nothing will happen. Instrumental rationality is the art of achieving your goals; it should be a familiar concept to Less Wrong readers. And self-identity, among other things, is about having those goals in the first place.
The spheres will never align entirely, and it is important to recognise that we are only aspiring rationalists, and to recognize and work around our weaknesses when they can't be easily fixed.
You don't have to lead other people to be a rational leader; you might only be leading yourself. But there's no reason to be afraid of it. I see true rationalists as making good leaders.
B. People stuff

In order to achieve your goals, it is likely you will need to interact with other people; to lead them, influence them, cooperate with them. You will also need to influence yourself; learn how to make yourself more effective. You may even need to go beyond basic individual interaction and deal with the issue of why people are the way they are.
This is something I believe may be a problem in the LW community: Not doing the people stuff.
C. Doing good

To give us a common goal, I want to find people who are interested in doing good. It doesn't have to be your only goal, and your definition of doing good doesn't have to be exactly the same as mine or anyone else's. It'll still be enough that we should co-operate.
If I could find such people, and if we could train ourselves and each other into being really effective, what would I see this organization doing?
1. Welcome

I would see us reaching out to altruistic individuals and organizations; finding people who are confused, who need help or who are still looking for the right approach. The idea is not to turn them all into rationalists, but rather to use our own rational skills to help and guide them. I would see this as our public face: the Altruist Support Network.
2. Think

Making a real positive difference in the world is hard. Even if you're motivated to do it, the infrastructure just isn't there to enable it. So we're going to need a lot of ideas, and ways to evaluate them. I feel certain that there are levers we can pull; small changes we can make that will have huge impacts, and that we can use rationality to help us find them. But I'll need your help.
3. Research

Some of the thinking has been done for us: there are papers and books already written, there are communities already out there. We need to find them - we need to create a good map of the rational-doing-good landscape. And then we need to push the boundaries, to create new knowledge.
4. Fund

If we have money, we want to spend it as wisely as we can: on organizations who share our goals and who have proven themselves to be among the most effective out there. Maybe we can tempt organizations into making changes with the prospect of a donation. And, very likely, we'll need to make money ourselves: to start a business and run it rationally, making a lot of profit and giving it to the causes we support. Such an endeavour sounds very difficult but worthwhile.
5. Act

Sometimes you just need to get out there and do things. Right now I don't know what; but this organization will not be an ivory tower. It exists to serve a purpose - making the world a better place - and we'll do what we need to in order to make that happen.
I apologize that my previous posts may have seemed a bit directionless. I hope this clears it up a little; I'm planning that my next bunch of posts will be sequence-style, gradually building up the ideas I've been having from foundations that are familiar.
The main things I want to know:
- Whether people see such an organization working
- Whether they see it as fundamentally different from anything which currently exists
- Whether they would want to be a part of it.
[Altruist Support] LW Go Foom
In which I worry that the Less Wrong project might go horribly right. This post belongs to my Altruist Support sequence.
Every project needs a risk assessment.
There's a feeling, just bubbling under the surface here at Less Wrong, that we're just playing at rationality. It's rationality kindergarten. The problem has been expressed in various ways:
- not a whole lot of rationality
- rationalist porn for daydreamers
- not quite as great as everyone seems to think
- shiny distraction
- only good for certain goals
And people are starting to look at fixing it. I'm not worried that their attempts - and mine - will fail. At least we'd have fun and learn something.
I'm worried that they will succeed.
What would such a Super Less Wrong community do? Its members would self-improve to the point where they had a good chance of succeeding at most things they put their mind to. They would recruit new rationalists and then optimize that recruitment process, until the community got big. They would develop methods for rapidly generating, classifying and evaluating ideas, so that the only ideas that got tried would be the best that anyone had come up with so far. The group would structure itself so that people's basic social drives - such as their desire for status - worked in the interests of the group rather than against it.
It would be pretty formidable.
What would the products of such a community be? There would probably be a self-help book that works. There would be an effective, practical guide to setting up effective communities. There would be an intuitive, practical guide to human behavior. There would be books, seminars and classes on how to really achieve your goals - and only the materials which actually got results would be kept. There would be a bunch of stuff on the Dark Arts too, no doubt. Possibly some AI research.
That's a whole lot of material that we wouldn't want to get into the hands of the wrong people.
Dangers include:
- Half-rationalists: people who pick up on enough memes to be really dangerous, but not on enough to realise that what they're doing might be foolish. For example, building an AI without adding the friendliness features.
- Rationalists with bad goals: Someone could rationally set about trying to destroy humanity, just for the lulz.
- Dangerous information discovered: e.g. the rationalist community develops a Theory of Everything that reveals a recipe for a physics disaster (e.g. a cheap way to turn the Earth into a block hole). A non-rationalist decides to exploit this.
If this is a problem we should take seriously, what are some possible strategies for dealing with it?
- Just go ahead and ignore the issue.
- The Bayesian Conspiracy: only those who can be trusted are allowed access to the secret knowledge.
- The Good Word: mix in rationalist ideas with do-good and stay-safe ideas, to the extent that they can't be easily separated. The idea being that anyone who understands rationality will also understand that it must be used for good.
- Rationality cap: we develop enough rationality to achieve our goals (e.g. friendly AI) but deliberately stop short of developing the ideas too far.
- Play at rationality: create a community which appears rational enough to distract people who are that way inclined, but which does not dramatically increase their personal effectiveness.
- Risk management: accept that each new idea has a potential payoff (in terms of helping us avoid existential threats) and a potential cost (in terms of helping "bad rationalists"). Implement the ideas which come out positive.
In the post title, I have suggested an analogy with AI takeoff. That's not entirely fair; there is probably an upper bound to how effective a community of humans can be, at least until brain implants come along. We're probably talking two orders of magnitude rather than ten. But given that humanity already has technology with slight existential threat implications (nuclear weapons, rudimentary AI research), I would be worried about a movement that aims to make all of humanity more effective at everything they do.
[Altruist Support] How to determine your utility function
Follows on from HELP! I want to do good.
What have I learned since last time? I've learned that people want to see an SIAI donation; I'll do it as soon as PayPal will let me. I've learned that people want more "how" and maybe more "doing"; I'll write a doing post soon, but I've got this and two other background posts to write first. I've learned that there's a nonzero level of interest in my project. I've learned that there's a diversity of opinions; it suggests if I'm wrong, then I'm at least wrong in an interesting way. I may have learned that signalling low status - to avoid intimidating outsiders - may be less of a good strategy than signalling that I know what I'm talking about. I've learned that I am prone to answering a question other than that which was asked.
Somewhere in the Less Wrong archives there is a deeply shocking, disturbing post. It's called Post Your Utility Function.
It's shocking because basically no-one had any idea. At the time I was still learning but I knew that having a utility function was important - that it was what made everything else make sense. But I didn't know what mine was supposed to be. And neither, apparently, did anyone else.
Eliezer commented 'in prescriptive terms, how do you "help" someone without a utility function?'. This post is an attempt to start to answer this question.
Firstly, what the utility function is and what it's not. It belongs to the field of instrumental rationality, not epistemic rationality; it is not part of the territory. Don't expect it to correspond to something physical.
Also, it's not supposed to model your revealed preferences - that is, your current behavior. If it did then it would mean you were already perfectly rational. If you don't feel that's the case then you need to look beyond your revealed preferences, toward what you really want.
In other words, the wrong way to determine your utility function is to think about what decisions you have made, or feel that you would make, in different situations. In other words, there's a chance, just a chance, that up until now you've been doing it completely wrong. You haven't been getting what you wanted.
So in order to play the utility game, you need humility. You need to accept that you might not have been getting what you want, and that it might hurt. All those little subgoals, they might just have been getting you nowhere more quickly.
So only play if you want to.
The first thing is to understand the domain of the utility function. It's defined over entire world histories. You consider everything that has happened, and will happen, in your life and in the rest of the world. And out of that pops a number. That's the idea.
This complexity means that utility functions generally have to be defined somewhat vaguely. (Except if you're trying to build an AI). The complexity will also allow you a lot of flexibility in deciding what you really value.
The second thing is to think about your preferences. Set up some thought experiments to decide whether you prefer this outcome or that outcome. Don't think about what you'd actually do if put in a situation to decide between them; then you will worry about the social consequences of making the "unethical" decision. If you value things other than your own happiness, don't ask which outcome you'd be happier in. Instead just ask, which outcome seems preferable?. Which would you consider good news, and which bad news?
You can start writing things down if you like. One of the big things you'll need to think about is how much you value self versus everyone else. But this may matter less than you think, for reasons I'll get into later.
The third thing is to think about preferences between uncertain outcomes. This is somewhat technical, and I'd advise a shut-up-and-multiply approach. (You can try and go against that if you like, but you have to be careful not to end up in weirdness such as getting different answers if you phrase something as one big decision or as a series of identical little decisions).
The fourth thing is to ask whether this preference system satisfies the von Neumann-Morgenstern axioms. If it's at all sane, it probably will. (Again, this is somewhat technical).
The last thing is to ask yourself: if I prefer outcome A over outcome B, do I want to act in such a way that I bring about outcome A? (continue only if the answer here is "yes").
That's it - you now have a shiny new utility function. And I want to help you optimize it. (Though it can grow and develop and change along with yourself; I want this to be a speculative process, not one in which you suddenly commit to an immutable life goal).
You probably don't feel that anything has changed. You're probably feeling and behaving exactly the same as you did before. But this is something I'll have to leave for a later post. Once you start really feeling that you want to maximize your utility then things will start to happen. You'll have something to protect.
Oh, you wanted to know my utility function? It goes something like this:
It's the sum of the things I value. Once a person is created, I value that person's life; I also value their happiness, fun and freedom of choice. I assign negative value to that person's disease, pain and sadness. I value concepts such as beauty and awesomeness. I assign a large bonus negative value to the extinction of humanity. I weigh the happiness of myself and those close to me more highly than that of strangers, and this asymmetry is more pronounced when my overall well-being becomes low.
Four points: It's actually going to be a lot more complicated than that. I'm aware that it's not quantitative and no terminology is defined. I'm prepared to change it if someone points out a glaring mistake or problem, or if I just feel like it for some reason. And people should not start criticizing my behavior for not adhering to this, at least not yet. (I have a lot of explaining still to do).
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)