You've had those moments -- the ones where you're very aware of where you're at in the world, and you're mapping out your future and plans very smartly, and you're feeling great about taking action and pushing important things forwards.

I used to find myself only reaching that place, at random, once or twice per year.

But every time I did, I would spend just a few hours sketching out plans, thinking about my priorities, discarding old things I used to do that didn't bring much value, and pushing my limits to do new worthwhile things. I thought, "This is really valuable. I should do this more often."

Eventually, I named that state: Reflective Control.

As often happens, by naming something it becomes easier to do it more often.

At this time, I still had a hazy poorly working feeling about what it was. So I tried to define it. After many attempts, I came to this:

> Reflective Control is when you're firmly off autopilot, in a high-positive and high-willpower state, and are able to take action.

You'll note there's four discreet components to it: firmly off autopilot (reflective), high positivity, high will, and cable of and oriented towards taking action.

I also asked myself, "How to know if you're in Reflective Control?"

My best answer of an exercise for it is,

> You set aside the impulses/distractions, and try to set a concrete Control-related goal. This is meta-work, meaning the process of defining your life and what needs to happen next. You do this calmly. By setting a concrete Control-related goal successfully and then executing on it, you know you're in an RC state.

> Example: "I will identify all the open projects I've got, and the next steps for each of them."

 

With that definition and that exercise in hand, I was able to do something which works almost magically when I wanted to take on big challenges: I could rate myself from 1-100 on the four key elements of the component, and then set a concrete goal to achieve, and analyze a little about which factor might be holding me back. Here is an example from my journal:

> Reflective 70/100, positive 70/100, will 65/100, action 40/100… ok, I'm feeling good once a good, just some anxiety suppressing will a little and action quite a bit, but no problem. My goal is to finish the xxx outline before I leave here.

I've found this incredibly useful. Summary:

*There's a state I call "Reflective Control" where I'm off autopilot and thinking (reflective), in a positive mood, with willpower and action-oriented.

*I can put explicit numbers on this, somewhat subjectively, from 1-100. This lets me see where the link in the chain is, if any.

*By setting a concrete goal and working towards it, you can get more objective feedback and balance whichever element is lowest with some practical actions.

Hundreds of Less Wrong posts summarize or repackage work previously published in professional books and journals, but Less Wrong also hosts lots of original research in philosophy, decision theory, mathematical logic, and other fields. This post serves as a curated index of Less Wrong posts containing significant original research.

Obviously, there is much fuzziness about what counts as "significant" or "original." I'll be making lots of subjective judgment calls about which suggestions to add to this post. One clear rule is: I won't be linking anything that merely summarizes previous work (e.g. Stuart's summary of his earlier work on utility indifference).

Update 09/20/2013: Added Notes on logical priors from the MIRI workshop, Cooperating with agents with different ideas of fairness, while resisting exploitation, Do Earths with slower economic growth have a better chance at FAI?

Update 11/03/2013: Added Bayesian probability as an approximate theory of uncertainty?, On the importance of taking limits: Infinite Spheres of Utility, Of all the SIA-doomsdays in the all the worlds...

Update 01/22/2014: Added Change the labels, undo infinitely good, Reduced impact AI: no back channels, International cooperation vs. AI arms race, Naturalistic trust among AIs: The parable of the thesis advisor’s theorem

---

General philosophy

• Highly Advanced Epistemology 101 for Beginners. Eliezer's bottom-up guide to truth, reference, meaningfulness, and epistemology. Includes practical applications and puzzling meditations.

• Seeing Red, A Study of Scarlet, Nature: Red in Tooth and Qualia. Orthonormal dissolves Mary's room and qualia.

• Counterfactual resiliency test for non-causal models. Stuart Armstrong suggests testing non-causal models for "counterfactual resiliency."

• Thoughts and problems with Eliezer's measure of optimization power. Stuart Armstrong examines some potential problems with Eliezer's concept of optimization power.

• Free will. Eliezer's particular compatibilist-style solution to the free will problem from reductionist viewpoint.

• The absolute Self-Selection Assumption. A clarification on anthropic reasoning, focused on Wei Dai s UDASSA framework.

• SIA, conditional probability, and Jaan Tallinn s simulation tree. Stuart Armstrong makes the bridge between Nick Bostrom s Self-Indication Assumption (SIA) and Jann Tallinn s of superintelligence reproduction.

• Mathematical Measures of Optimization Power. Alex Altair tackles one approach to mathematically formalizing Yudkowsky s Optimization Power concept.

• Caught in the glare of two anthropic shadows. Stuart_Armstrong provides a detailed analysis of the "anthrophic shadow" concept and its implications.

• Bayesian probability as an approximate theory of uncertainty?. Vladimir Slepnev argues that Bayesian probability is an imperfect approximation of what we want from a theory of uncertainty.

• Of all the SIA-doomsdays in the all the worlds.... Stuart_Armstrong on the doomsday argument, the self-sampling assumption and the self-indication assumption.

Giles, and some others, have asked questions about donating to one or more of CEA’s sub-organisations. In what follows, I address these questions. I felt it would be clearest for me to mainly cluster questions under general headings, rather than address the specific wording of every question. (Note: thanks to help from Ben Todd on this!)

A couple of clarifications

Centre for Effective Altruism is a legal entity that comprises 4 organisations: Giving What We Can, 80,000 Hours, Effective Animal Activism, and The Life You Can Save. EAA is formally still a sub-project of 80,000 Hours, but should be thought of as separate for accounting purposes and may well become a separate organisation. In the previous blog post I talked about GWWC and 80k only, but because there’s been interest, here I’ll discuss the other two as well, albeit more briefly.

Some numbers follow. These are true as of Nov 20th 2012 (or, rather, are best estimates as of that date), but are very likely to change in the near future. They should therefore be taken as illustrations merely.

Finally, because all the organisations that comprise CEA are young, there are certain policy issues that still have not been decided upon; and some that have been decided upon may change in the near future as we learn. Where possible, I have tried to flag which policies are as yet undecided.

What’s your expenditure?

Below are rough estimates for expenditure from start Q2 2012 (when we first took staff) until end Q4 2013 (our short-term fundraising horizon). All numbers are in thousands.

80k (basic): £118.8 = ~$190

80k (inc. some expansion): £139.1 = ~$220

GWWC (basic): £87.4 = ~$140

GWWC (inc. some expansion): £107.7 = ~$170

LYCS (basic): £47.88 = ~$76

EAA (basic): £32.76 = ~$52

The large majority of our expenditure is on staff. 80k (basic) comprises one full-time staff member from Q2 2012, two staff members working 0.4 time from Q2 2012, and one full-time staff member from Q1 2013. GWWC (basic) comprises one 5/8 staff member from Q2 2012, and two staff members working 0.6 time from Q2 2012. LYCS (basic) comprises one full-time staff member from Q2 2013, and some money earmarked for on-line marketing by one funder. EAA (basic) comprises one full-time staff member from Q1 2013. Across CEA, we typically employ one intern-year for every employee-year.

For both GWWC and 80k, the difference between the ‘basic’ scenario and the ‘expansion’ scenario is that we would hire one additional person from Q3 2013, and employ one additional intern-year for 2013. The ‘expansion’ scenario indicates a cautious limit on our use for more funding; though it certainly seems to us that we could spend money well above that amount, we would need to discuss whether it could be detrimental in the long-run for the organisations to grow that fast. We could very comfortably spend within the ‘expansion’ scenario, and would feel hindered if we were not able to spend up to that amount.

Each employee is paid a starting salary of £18 per annum, significantly below market rates for graduates even within the not-for-profit world, and without accounting for the fact that our employees are significantly more qualified than the average graduate. Interns are unpaid, but are typically given expenses. A significant proportion of our labour is still voluntary.

What’s your income? What’s your shortfall?

All numbers are now in $000s. I'll measure 'shortfall' relative to the 'basic' budget. The fundraising numbers below include both income that we have already received and income that we expect to receive, discounted according to a conservative estimate of its likelihood (50%). I’ll assume that CEA unrestricted money is divided as follows: 0.4 to 80k, 0.4 to GWWC, 0.12 to LYCS and 0.08 to EAA. (This division does not represent a policy about how we divide unrestricted funds. Currently that policy is not yet determined, so I’ve chosen these numbers as illustrative. More on use of unrestricted money in “earmarking and fungibility” below.)

80k raised: 85.1. Shortfall = 190 - 85.1 = ~$105

GWWC raised: 117.9. Shortfall = 140 – 117.9 = ~$22

LYCS raised: 63.4. Shortfall = 76 – 63.4 = ~$12.5

EAA raised: 27.2. Shortfall = 52 – 27.2 = ~$25

For both GWWC and 80k, to get the shortfall for the 'expansion' budget – which represents spending which we could easily accommodate without sacrificing quality of work —  add 30 to the 'shortfall' number.

How would you spend additional money?

As the numbers above suggest, in most cases additional donations would be spent on basic costs — principally, paying staff — over 2013. If GWWC or 80k exceeded their ‘basic’ budget, then additional money would be put towards on their ‘additional’ budget: principally, hiring one new staff member each.  The desired marginal hire for GWWC is a Communications Director.  The desired marginal hire for 80,000 Hours is a Careers Researcher and Adviser.

Where’s that income from?

Our donations come from a variety of sources. Private donors, of varying degrees of wealth, make up the large majority of our income. GWWC has received a grant from one foundation. The majority of donations come from within the effective giving community, though a sizable proportion comes from outside that community and we’re actively pursuing further leads there, including high net worths. If we ever had a significant donation commitment that used up our room for more funding, we’d let other donors know immediately.

Room for More Funding, and a Co-ordination Problem

Suppose that, within the effective giving community, there is $N that people would want to donate to CEA, conditional on CEA having room for more funding. But CEA only had room for $M, where M<N. Every giver thinks, “well, CEA is going to reach its room or more funding anyway. So there’s no reason why I should given.” So no-one gives to CEA. That would be a bad outcome on our part. Alternatively, perhaps every giver thinks, “well, I’ll just give to CEA anyway”. So CEA receives $N, whereas it can only spend $M well, and there is an $N-$M excess. That would be a bad outcome on the part of the giver. So what’s the solution?

One solution to that problem (though I haven't thought about it that much) is as follows: we decide upon a funding limit for each organisation. We say that if we receive donations above that limit (before a designated time), we will donate the excess to the most cost-effective charities. Different givers think that different causes are the most important, so we’ll donate to the different cause areas depending on what proportion of CEA donations would have been given to those causes if they hadn’t been donated to CEA. So if we received 70% of donations that would have been donated to global poverty, we’ll give 70% of the excess to AMF; similarly for animal welfare and x-risk.

The advantages of this are as follows. It safeguards against CEA having less money than it needs because of the co-ordination problem. It ensures that givers can donate to CEA while knowing that the money won’t be spent on CEA above its room for more funding. It’s also what GiveWell does, and insofar as GW appear to us to be a very well-run organisation, it’s worth imitating them.

What are the alternative solutions? Well, we could bank the money and use it the following year. So the excess money donated to CEA is used one year later than the giver might have expected, and we spend less time on fundraising for the following year. Or we could go ‘first come first served’: we keep accepting donations until we hit our RFMF. I think that the latter suggestion is a bad one. The former is potentially good I think, and simple, and I’m open to comments on which solution potential donors think is preferable.

Different Cause Areas

GWWC and LYCS are focused on global poverty, and have no plans to change that. EAA is focused on animal welfare. 80,000 Hours is open to any plausibly high-impact activity. There is currently no organisation within CEA dedicated purely to x-risk mitigation, but, given demand, it’s not unlikely that one will be created in the mid-term future.

“Earmarking” and Fungibility

Some comments mentioned “earmarking”. I think that’s a misleading term in this context. “Earmarking” normally refers to donations that are tied to a specific activity. Whereas, when one donates to GWWC, the donation is not tied to a specific activity. CEA shouldn’t be thought of as an organization over and above the four organisations.

We actively encourage donations that are restricted to one organization only, if you think that one organization is more cost-effective than the others. In order to avoid the fungibility problem, I considered asking only for restricted donations. It seems to me on balance that the costs of this policy outweigh the benefits, but I’m not sure.

My current preferred solution is as follows. Every 6 months, after the reviews of each organization (see next section), the trustees decide how to allocate unrestricted funding. The default they use is that unrestricted funding is allocated in proportion with restricted funding. If this default holds — either exactly or approximately — then fungibility of donations is not an issue. In fact, if the default holds, then fungibility is negative: donating $1 to GWWC would move slightly more than $1 to GWWC, because it would also increase the proportion of CEA unrestricted money that it receives. The trustees deviate from this default if there are compelling reasons for doing so (e.g. a major donor for one organization unexpectedly drops out, rendering basic expenditure uncertain). In the long run (and ex ante), we wouldn’t expect these deviations to favour one organization over another, so, in the long run (and ex ante), fungibiltiy is again not an issue. Moreover, from this arrangement we would expect each organization to benefit in terms of financial stability and from the success of their sister organisations.

Self-Evaluation and Impact Assessment

I’ll describe 80k’s process. GWWC’s is very similar.

Every 6 months, 80k will have to write a report of its progress over the last 6 months, including achievements and failures, how its progress compares to the goals stated 6 months ago, and write concrete, measurable goals for the next 6 months. This report will then be reviewed by two boards. The trustees of CEA: myself, Nick Beckstead, and Toby Ord. And an "Advisory Committee", consisting of 80k supporters (and often donors) who aren't in any way involved with the running of 80k. The Executive Director of 80k (and one or two others) will meet with these boards, and they'll discuss the report. Each board will write a summary of conclusions. All three documents (initial report, and two commentaries from the boards) will be posted on the blog.

Every year (probably in spring or early summer - a quiet time for us), we'll complete a more in-depth impact-evaluation, at least in terms of money moved, person-hours moved, money pledged and person-hours pledged. All-year round, we measure progress with respect to pre-chosen goals. The ED of 80k sends progress reports to the 80k team every week. Currently, because marketing and recruitment are our key priorities, our principal metrics are number of new members per week, % of members who say that they’ve changed their career plans because of 80k, income pledged per member, unique visitors to the website, and number of advising sessions given.

Miscellaneous CEA Questions

Which is more useful, regular donations or lump sums?

Either is good. Most charities prefer regular donations because people are likely to give more that way (they forget about the direct debit). But I'd rather you were giving on the basis of perceived cost-effectiveness, rather than status quo bias! Financial forecasting is really important for us, though, so if it's lump sums, we really appreciate knowing the chance they'll be repeated in future years. And we have a steep discount rate (perhaps 20%?) so we greatly prefer money sooner rather than later.

If you had funds to hire an extra person, do you know how that person would be? How important is it to find talented people to work for you?  Are you trying to find someone from the top 5%? The top 1%?

It's difficult to give a meaningful reply to that — 1% in terms of general ability, or fit for us? I'll answer for 'general ability' (whatever that means).  We're generally selecting only from top universities, which filters out a large majority of the population. As an approximation (but merely a very rough approximation): There are roughly 772,000 18 yr olds in the UK, of which 7000 go to Oxford or Cambridge. We mainly select from those universities (or equivalent standard elsewhere), so that already filters out 99% of the population.

Within such universities, we normally recruit very high-performing graduates — perhaps in the top 10% or 5%. Which would suggest that we're recruiting from the top 0.1% from the population. But, like I say, I'm not sure that that number is that meaningful. I'm not certain why, but we do seem to be able to recruit exceptionally talented people. (Like Niel, who's starting with us from January).

How much personal connection and communication is there between CEA and these orgs?
- THINK
- Global Catastrophic Risk Institute
- Center for Applied Rationality
- Future of Humanity Institute
- GiveWell

Lots. Mark Lee, founder of THINK, came through GWWC. I gave a talk for the Brown THINK chapter the other day, and helped their co-President with plans for the year. We've met a couple of times with Seth Baum. We know Julia and Anna well, and support CFAR. Toby Ord (a trustee of CEA) is a research associate at FHI, and Will participates in FHI events. We're in regular contact with GiveWell. A core CEA volunteer is considering working for them.

I could say much more, but it would get long-winded. We support all the above organisations, and aim to co-ordinate with them all, so that we don't get in each others' way, and can help each other out.

Where do you see the delineation between what CEA does and what other effective altruist orgs do?

We worry a lot about needlessly doubling up on or competing with work done by other effective altruist organisations. Taking our four organisations in turn:

GWWC: Along with LYCS, the only group in the world promoting major individual cost-effective giving. Does charity effectiveness research, but only where we think we can usefully add to what Givewell does.

80k: The only effective altruist organisation doing careers advice. The most broad focused effective altruist organisation except for THINK, but we’re distinct from THINK in that we provide careers advice through web content and one-on-one sessions rather than setting up meet-ups.

EAA: The only animal focused effective altruist organisation (EAA is effectively doing what 80k would have done in this area, except we thought it was useful to give it separate branding.

TLYCS: Similar in aims to GWWC, but lower-bar entry for most members. Planning different outreach routes

What are you planning in the way of financial transparency?

We'll publish an annual financial report, with a breakdown of costs. We’d like to be able to regularly explain room for more funding (etc.) as we do above, but doing so uses considerable time of high-level people within the organization, so we can’t promise that.  In general, we take GiveWell as a model organisation, and will often emulate their practices.

Miscellaneous GWWC Question

You said in your LW post that you have "much more information available" on GWWC's impact.

Yes. If you email me (will [dot] crouch [at] 80000hours.org), you can see the calculations by which we estimated GWWC’s impact.

Miscellaneous 80k Questions

You do a bunch of different but related things - website content, speaker events, career counseling,
- Do you imagine yourself specialising in just one of these in the future?
- Are you at the stage of experimenting to find out which activity is the most effective?
- Is there synergy between them? (e.g. if career advice sessions and website content are both a lot more effective if you're also doing the other one)

We’re creating a new type of organisation, and there’s a great deal we don’t know. We see our priority as testing these different approaches and improving them. Having a basket of methods lets us gain more information, and prevents us from stalling if one method turns out not to work.

For each method, we track a bunch of metrics which ultimately relate to our bottom line: resources shifted to the most effective causes that wouldn’t have been shifted otherwise. We propose tests for ways to improve these metrics. If our hypotheses about what we expect to work are disconfirmed, we change our approach. Otherwise, we move to scale up the method.

Whether we end up specialising, therefore, depends on whether one of the methods ends up being significantly more effective than the rest. And whatever happens, since we’re constantly seeking to improve, I imagine we’ll always be experimenting with new processes.

At the moment, we broadly see the web content and one-on-one advice as our most important services, and we’re expecting to scale them the most (though there’s a lot of flexibility within where we take these). We pursue some lectures, especially in Oxford, since they have high marginal returns, but we don’t currently expect to scale them. We’re exploring some other methods, but have not tested them yet.

I have very little idea about what the 80K community is like or how exactly you invest in it
- in what ways does your team interact with your community, other than one-on-one career advice and hosting speaker events?
- do you invest in members' skills such as critical thinking and the ability to evaluate organisations?
- what other skills and qualities do you want to develop in your members, and how do you plan to go about it?
- to what extent do you think talents and abilities are inherent (or at least beyond your control), and to what extent are they trainable?

Probably one of the most useful things we do is forge links between people in the community who can help each other out. For instance, we’ve introducing people who have successfully navigated applications to finance jobs to others who want to do the same. We brought together a bunch of people interested in the animal cause to set up EAA. We’ve introduced people who are in the same research field. At the minute, this mainly happens via personal introductions, but we’re developing tools to make this easier online.
Besides this, the team interacts with the community via the members’ googlegroup and our online discussion forum, 80000hours.org/discussion.

Our current focus in our providing our members with really useful information about which career they should enter in order to have the most impact. Our main way of improving their skills is by introducing them to mentors. We also sometimes coach people through tough career stages in our one-on-one advice (e.g. we recently helped get one of our interns a Marshall scholarship). Longer term, we might switch to have a greater focus on self-improvement, but that niche has a lot more competition (e.g. if you care about becoming more rational, go to Less Wrong).

Is 80K planning activity in any new physical locations?
- If so, where?
- If there's already a THINK community in that location, what do you imagine the relationship between THINK and 80K looking like?

We encourage any 80k members who’d like to start an 80k meet-up to do so, and we’ll happily give them advice and support; but we don’t plan to invest significant staff time in setting up new physical locations. This is because our current focus is on our web content and one-on-one advice. The exception is that we’d like to intensify existing involvement in Oxford, Yale and Princeton, which we see as test grounds and already have some infrastructure.

We could easily change our minds, however, and we’re pretty interested in the idea of doing a lecture tour (which we could support with the web content and one-on-one advice).  We wouldn’t set up an 80k presence in a new location, as opposed to sending people to THINK, unless we thought that there was good reason that having two organisations in the area would be more effective.

There's apparently a lot of interest in x-risk among 80K members. Do you know why this is?

I think that the 34% is higher than the percentage of x-risk concerned members in the long-run (which I'd guess will end up at about 10%), due to initial selection effects (we got quite a few members from the x-risk network, and others who would have been GWWC members were if not for the fact that they were principally concerned about x-risk). However, there does seem to be a strong positive correlation between how dedicated members are, and whether they are concerned by x-risk. And we haven’t yet really discussed x-risk as a cause area. So it’s difficult to say what proportion of resources we generate will be x-risk focused.

A number of people are convinced by x-risk but just don't think that there currently exists a good enough giving opportunity. So the proportion would increase considerably if a really clear x-risk giving opportunity arose (e.g. if GW ever recommended an x-risk org).

Do you know of any other organisations that do anything similar to what you do? (other than ones I've already mentioned). In particular any groups that give career advice to philanthropists.

As far as we can tell, there’s no-one else providing careers advice focused on how you can make a difference. All that exists is informal advice about impact given by friends and within other effective altruist communities (e.g. LW, GWWC).
Making a difference aside, it seems like the average quality of careers advice in general is pretty low, and rarely evidence-based or aware of decision making biases.

From several perspectives, we think we’re in a very interesting market niche. As far as philanthropy goes, there are some groups focused on fostering it e.g. http://youngphilanthropy.org.uk/, but they tend to have little focus on effectiveness.

Miscellaneous EAA Questions

What are the plans for EAA? When will it be spun off? Is there much interest in it from new members of 80k? Is anybody other than Eitan Fischer (who's in school) working on it at the moment?

EAA is taking on a full-time Executive Director some time between Jan and Sept. The initial priorities will be scaling up the charity effectiveness research, taking it to major philanthropists and fundraising. If EAA achieves enough scale, we’ll promote it to a full member of CEA.

Around 10-20% of our new members are interested in the animal cause (and we haven’t promoted it much directly, beyond a couple of blog posts). They tend to make use of EAA. Eitan has a small team of volunteers and advisers helping him out part-time. There’s probably about 4-8 people involved in some capacity at most times. This should increase significantly once we recruit an Executive Director.

The unprecedented gap in Methods of Rationality updates prompts musing about whether readership is increasing enough & what statistics one would use; I write code to download FF.net reviews, clean it, parse it, load into R, summarize the data & depict it graphically, run linear regression on a subset & all reviews, note the poor fit, develop a quadratic fit instead, and use it to predict future review quantities.

Then, I run a similar analysis on a competing fanfiction to find out when they will have equal total review-counts. A try at logarithmic fits fails; fitting a linear model to the previous 100 days of _MoR_ and the competitor works much better, and they predict a convergence in <5 years.

Master version: http://www.gwern.net/hpmor#analysis