The Future of Humanity Institute has released video footage of the 2012 Winter Intelligence Conference.  The videos currently available are:

• Stuart Armstrong - Predicting AI... or Failing to
• Miles Brundage - Limitations and Risks of Machine Ethics
• Steve Omohundro - Autonomous Technology and the Greater Human Good
• Anders Sandberg - Ethics and Impact of Brain Emulations
• Carl Shulman - Could We Use Untrustworthy Human Brain Emulations to Make Trustworthy Ones

In a recent essay, Brian Tomasik argues that meme-spreading has higher expected utility than x-risk reduction. His analysis assumes a classical utilitarian ethic, but it may be generalizable to other value systems.  Here's the summary:

I personally do not support efforts to reduce extinction risk because I think space colonization would potentially give rise to astronomical amounts of suffering. However, even if I thought reducing extinction risk was a good idea, I would not work on it, because spreading your particular values has generally much higher leverage than being one more voice for safety measures against extinction in a world where reducing extinction risk is hard and almost everyone has some incentives to invest in the issue.

Added February 2014: As I no longer have time to keep both posts updated, all further updates will be made to the version of this post hosted on my personal blog.

In a recent post, Qiaochu Yuan noted that "various mnemonic techniques like memory palaces, along with spaced repetition, seem to more or less solve the problem of memorization." The list below is an attempt to compile all existing Anki decks created by Less Wrong users, in the hope that they will be of help to others in memorizing the corresponding material. (Anki is arguably the most popular spaced repetition software.)  If you know of a deck not included here, please mention it in the comments section and I'll add it to the list. Thanks!

Brian Tomasik's latest article, 'Quantify with Care', seems to be of sufficient interest to readers of this forum to post a link to it here.  Abstract:

Quantification and metric optimization are powerful tools for reducing suffering, but they have to be used carefully. Many studies can be noisy, and results that seem counterintuitive may indeed be wrong because of sensitivity to experiment conditions, human error, measurement problems, or many other reasons. Sometimes you're looking at the wrong metric, and optimizing a metric blindly can be dangerous. Designing a robust set of metrics is actually a nontrivial undertaking that requires understanding the problem space, and sometimes it's more work than necessary. There can be a tendency to overemphasize statistics at the expense of insight and to use big samples when small ones would do. Finally, think twice about complex approaches that sound cool or impressive when you could instead use a dumb, simple solution.

I'm posting this article on behalf of Brian Tomasik, who authored it but is at present too busy to respond to comments.

Update from Brian: "As of 2013-2014, I have become more sympathetic to at least the spirit of CEV specifically and to the project of compromise among differing value systems more generally. I continue to think that pure CEV is unlikely to be implemented, though democracy and intellectual discussion can help approximate it. I also continues to feel apprehensive about the conclusions that a CEV might reach, but the best should not be the enemy of the good, and cooperation is inherently about not getting everything you want in order to avoid getting nothing at all."

Introduction

I'm often asked questions like the following: If wild-animal suffering, lab universes, sentient simulations, etc. are so bad, why can't we assume that Coherent Extrapolated Volition (CEV) will figure that out and do the right thing for us?

Disclaimer

Most of my knowledge of CEV is based on Yudkowsky's 2004 paper, which he admits is obsolete. I have not yet read most of the more recent literature on the subject.

Reason 1: CEV will (almost certainly) never happen

CEV is like a dream for a certain type of moral philosopher: Finally, the most ideal solution for discovering what we really want upon reflection!

The fact is, the real world is not decided by moral philosophers. It's decided by power politics, economics, and Darwinian selection. Moral philosophers can certainly have an impact through these channels, but they're unlikely to convince the world to rally behind CEV. Can you imagine the US military -- during its AGI development process -- deciding to adopt CEV? No way. It would adopt something that ensures the continued military and political dominance of the US, driven by mainstream American values. Same goes for China or any other country. If AGI is developed by a corporation, the values will reflect those of the corporation or the small group of developers and supervisors who hold the most power over the project. Unless that group is extremely enlightened, CEV is not what we'll get.

Anyway, this is assuming that the developers of AGI can even keep it under control. Most likely AGI will turn into a paperclipper or else evolve into some other kind of Darwinian force over which we lose control.

Objection 1: "Okay. Future military or corporate developers of AGI probably won't do CEV. But why do you think they'd care about wild-animal suffering, etc. either?"

Well, they might not, but if we make the wild-animal movement successful, then in ~50-100 years when AGI does come along, the notion of not spreading wild-animal suffering might be sufficiently mainstream that even military or corporate executives would care about it, at least to some degree.

If post-humanity does achieve astronomical power, it will only be through AGI, so there's high value for influencing the future developers of an AGI. For this reason I believe we should focus our meme-spreading on those targets. However, this doesn't mean they should be our only focus, for two reasons: (1) Future AGI developers will themselves be influenced by their friends, popular media, contemporary philosophical and cultural norms, etc., so if we can change those things, we will diffusely impact future AGI developers too. (2) We need to build our movement, and the lowest-hanging fruit for new supporters are those most interested in the cause (e.g., antispeciesists, environmental-ethics students, transhumanists). We should reach out to them to expand our base of support before going after the big targets.

Objection 2: "Fine. But just as we can advance values like preventing the spread of wild-animal suffering, couldn't we also increase the likelihood of CEV by promoting that idea?"

Sure, we could. The problem is, CEV is not an optimal thing to promote, IMHO. It's sufficiently general that lots of people would want it, so for ourselves, the higher leverage comes from advancing our particular, more idiosyncratic values. Promoting CEV is kind of like promoting democracy or free speech: It's fine to do, but if you have a particular cause that you think is more important than other people realize, it's probably going to be better to promote that specific cause than to jump on the bandwagon and do the same thing everyone else is doing, since the bandwagon's cause may not be what you yourself prefer.

Indeed, for myself, it's possible CEV could be a net bad thing, if it would reduce the likelihood of paperclipping -- a future which might (or might not) contain far less suffering than a future directed by humanity's extrapolated values.

Reason 2: CEV would lead to values we don't like

Some believe that morality is absolute, in which case a CEV's job would be to uncover what that is. This view is mistaken, for the following reasons: (1) Existence of a separate realm of reality where ethical truths reside violates Occam's razor, and (2) even if they did exist, why would we care what they were?

Yudkowsky and the LessWrong community agree that ethics is not absolute, so they have different motivations behind CEV. As far as I can gather, the following are two of them:

Motivation 1: Some believe CEV is genuinely the right thing to do

As Eliezer said in his 2004 paper (p. 29), "Implementing CEV is just my attempt not to be a jerk." Some may believe that CEV is the ideal meta-ethical way to resolve ethical disputes.

I have to differ. First, the set of minds included in CEV is totally arbitrary, and hence, so will be the output. Why include only humans? Why not animals? Why not dead humans? Why not humans that weren't born but might have been? Why not paperclip maximizers? Baby eaters? Pebble sorters? Suffering maximizers? Wherever you draw the line, there you're already inserting your values into the process.

And then once you've picked the set of minds to extrapolate, you still have astronomically many ways to do the extrapolation, each of which could give wildly different outputs. Humans have a thousand random shards of intuition about values that resulted from all kinds of little, arbitrary perturbations during evolution and environmental exposure. If the CEV algorithm happens to make some more salient than others, this will potentially change the outcome, perhaps drastically (butterfly effects).

Now, I would be in favor of a reasonable extrapolation of my own values. But humanity's values are not my values. There are people who want to spread life throughout the universe regardless of suffering, people who want to preserve nature free from human interference, people who want to create lab universes because it would be cool, people who oppose utilitronium and support retaining suffering in the world, people who want to send members of other religions to eternal torture, people who believe sinful children should burn forever in red-hot ovens, and on and on. I do not want these values to be part of the mix.

Maybe (hopefully) some of these beliefs would go away once people learned more about what these wishes really implied, but some would not. Take abortion, for example: Some non-religious people genuinely oppose it, and not for trivial, misinformed reasons. They have thought long and hard about abortion and still find it to be wrong. Others have thought long and hard and still find it to be not wrong. At some point, we have to admit that human intuitions are genuinely in conflict in an irreconcilable way. Some human intuitions are irreconcilably opposed to mine, and I don't want them in the extrapolation process.

Motivation 2: Some argue that even if CEV isn't ideal, it's the best game-theoretic approach because it amounts to cooperating on the prisoner's dilemma

I think the idea is that if you try to promote your specific values above everyone else's, then you're timelessly causing this to be the decision of other groups of people who want to push for their values instead. But if you decided to cooperate with everyone, you would timelessly influence others to do the same.

This seems worth considering, but I'm doubtful that the argument is compelling enough to take too seriously. I can almost guarantee that if I decided to start cooperating by working toward CEV, everyone else working to shape values of the future wouldn't suddenly jump on board and do the same.

Objection 1: "Suppose CEV did happen. Then spreading concern for wild animals and the like might have little value, because the CEV process would realize that you had tried to rig the system ahead of time by making more people care about the cause, and it would attempt to neutralize your efforts."

Well, first of all, CEV is (almost certainly) never going to happen, so I'm not too worried. Second of all, it's not clear to me that such a scheme would actually be put in place. If you're trying to undo pre-CEV influences that led to the distribution of opinions to that point, you're going to have a heck of a lot of undoing to do. Are you going to undo the abundance of Catholics because their religion discouraged birth control and so led to large numbers of supporters? Are you going to undo the over-representation of healthy humans because natural selection unfairly removed all those sickly ones? Are you going to undo the under-representation of dinosaurs because an arbitrary asteroid killed them off before CEV came around?

The fact is that who has power at the time of AGI will probably matter a lot. If we can improve the values of those who will have power in the future, this will in expectation lead to better outcomes -- regardless of whether the CEV fairy tale comes true.

Anders Sandberg, Where are they?, TEDxUHasselt.

On the long term, how much change in the universe can a civilization possibly cause? In this talk, Anders Sandberg brings an enthusiastic introduction to the different scenarios of the Fermi paradox and what they mean for the future of humanity.

Discussion article for the meetup : Fourth Buenos Aires Less Wrong meetup

Discussion article for the meetup : Fourth Buenos Aires Less Wrong meetup

Here. Some excerpts:

What do you think are some good heuristics for doing high impact research?

One idea is to go for under-researched fields. Progress in a field is typically a very convex learning curve: rapid progress at first when the low-hanging fruits get picked by the pioneers, followed by slowing progress as the problems get harder and it takes longer to learn the necessary skills to get to them. So the same amount of effort might produce far more progress in a little studied field than in a big one. [...]

It can also help to turn the question around: what aspects of human life matter? Looking at human life, we sleep about a third of the time, and there’s very little research into how to enhance sleep. Understanding the health effects of what we eat is probably worth billions of pounds per year. But there are no financial incentives here. Maybe a simple approach for finding high impact research areas might be to look at the most common google searches: you can get a pretty good idea of what human behaviour matters a lot!

Do you think it’s better to be a generalist and get a broad understanding of a lot of things, or to specialise early and really focus on a single area you think is high impact?

Over the history of my academic career my most useful courses have been linear algebra, all the statistics and probability theory I’ve been able to pick up, some basic computer science, and a course on natural disasters. [...]

Even if you do focus on one field, knowing enough about other fields is good as you can recognise when you need the help of someone from another department.

What other barriers are there to doing important research?

Looking at some of these under-researched fields, the problem is that a lot of them don’t even exist as fields. Typically you’re unlikely to get funding in unknown fields as well: unless you’re a really good salesman! So one heuristic would be to look at the topics you know, do a matrix and look at the interactions: which areas do you see that have nobody doing anything in?

When I went to a computational neuroscience conference last year, I was slightly depressed as I saw a poster which was exactly the same research topic as my last poster! It was pretty clear the young grad student had reached the same conclusion I did, and had never heard of my research which was published 6 years ago! Many fields have this problem that they don’t have very much of a memory, which affects progress.

Fields like AI are struggling because there’s no good way of comparing progress. How much smarter are current general AI programs than some of the classics? Nobody knows, and you can’t test the older programs because the source code and everything has been lost except a few bizarre papers from the early 70s.

Bias blind spot.

Sophistication effect. [also check discussion 

Naive cynicism

Both GiveWell and the Centre for Effective Altruism (CEA) --an Oxford-based umbrella organization consisting of Giving What We Can, 80,000 Hours, The Life You Can Save, and Effective Animal Activism-- have been discussed here before.  So I thought some folks might want to know that these organizations are recruiting for a number of positions.  Here are relevant excerpts from the official job announcements:

GiveWell: Research Analyst

GiveWell is looking for a Research Analyst to help us evaluate charities, find the most outstanding giving opportunities, and publish our analysis to help donors decide where to give.

 

Effective Animal Activism: Executive Director

Effective Animal Activism is a recently-founded project of 80,000 Hours. It is the world’s first online resource and international community for people who want to reduce animal suffering effectively. We are currently looking for a part-time executive director. Responsibilities will include creating content, managing the community, publicizing the site, and overseeing as well as undertaking further charity research. Future projects include creating a publication on our intervention evaluation once complete, attending conferences, running ad campaigns, and reaching out to the media, animal charities and philanthropists.

 

Giving What We Can: Head of Communications

We are looking for someone to communicate Giving What We Can’s message to the world. As Communications Manager you would be responsible for handling our press relations and guiding our public image.

 

80,000 Hours: Head of Careers Research

We are looking for someone to drive cutting-edge research into effective ethical careers and translate it into one-on-one and online careers advice, which you’ll share with interesting people from all over the world.

 

The Life You Can Save: Director of Outreach (Intern)

We are looking for someone to lead our outreach to pledgers and supporters as well as local groups, other charities, and corporations. In this role, you’ll play a key part in setting our strategic priorities and driving the growth of The Life You Can Save. You’ll be working alongside Peter Singer – one of the most influential ethicists of the 20th century.

 

Centre for Effective Altruism: Head of Fundraising and External Relations

We are looking for someone to manage our fundraising and represent us to other organisations. In this role you would serve all four organisations in the Centre for Effective Altruism: Giving What We Can, 80,000 Hours, The Life You Can Save and Effective Animal Activism.

(Full disclosure: I'm friends with the co-founders of CEA and have donated to Effective Animal Activism.)