LessWrong has twice discussed the PhilPapers Survey of professional philosophers' views on thirty controversies in their fields — in early 2011 and, more intensively, in late 2012. We've also been having some lively debates, prompted by LukeProg, about the general value of contemporary philosophical assumptions and methods. It would be swell to test some of our intuitions about how philosophers go wrong (and right) by looking closely at the aggregate output and conduct of philosophers, but relevant data is hard to come by.

Fortunately, Davids Chalmers and Bourget have done a lot of the work for us. They released a paper summarizing the PhilPapers Survey results two days ago, identifying, by factor analysis, seven major components consolidating correlations between philosophical positions, influences, areas of expertise, etc.

1. Anti-Naturalists: Philosophers of this stripe tend (more strongly than most) to assert libertarian free will (correlation with factor .66), theism (.63), the metaphysical possibility of zombies (.47), and A theories of time (.28), and to reject physicalism (.63), naturalism (.57), personal identity reductionism (.48), and liberal egalitarianism (.32).

Anti-Naturalists tend to work in philosophy of religion (.3) or Greek philosophy (.11). They avoid philosophy of mind (-.17) and cognitive science (-.18) like the plague. They hate Hume (-.14), Lewis (-.13), Quine (-.12), analytic philosophy (-.14), and being from Australasia (-.11). They love Plato (.13), Aristotle (.12), and Leibniz (.1).

2. Objectivists: They tend to accept 'objective' moral values (.72), aesthetic values (.66), abstract objects (.38), laws of nature (.28), and scientific posits (.28). Note 'Objectivism' is being used here to pick out a tendency to treat value as objectively binding and metaphysical posits as objectively real; it isn't connected to Ayn Rand.

A disproportionate number of objectivists work in normative ethics (.12), Greek philosophy (.1), or philosophy of religion (.1). They don't work in philosophy of science (-.13) or biology (-.13), and aren't continentalists (-.12) or Europeans (-.14). Their favorite philosopher is Plato (.1), least favorites Hume (-.2) and Carnap (-.12).

3. Rationalists: They tend to self-identify as 'rationalists' (.57) and 'non-naturalists' (.33), to accept that some knowledge is a priori (.79), and to assert that some truths are analytic, i.e., 'true by definition' or 'true in virtue of 'meaning' (.72). Also tend to posit metaphysical laws of nature (.34) and abstracta (.28). 'Rationalist' here clearly isn't being used in the LW or freethought sense; philosophical rationalists as a whole in fact tend to be theists.

Rationalists are wont to work in metaphysics (.14), and to avoid thinking about the sciences of life (-.14) or cognition (-.1). They are extremely male (.15), inordinately British (.12), and prize Frege (.18) and Kant (.12). They absolutely despise Quine (-.28, the largest correlation for a philosopher), and aren't fond of Hume (-.12) or Mill (-.11) either.

4. Anti-Realists: They tend to define truth in terms of our cognitive and epistemic faculties (.65) and to reject scientific realism (.6), a mind-independent and knowable external world (.53), metaphysical laws of nature (.43), and the notion that proper names have no meaning beyond their referent (.35).

They are extremely female (.17) and young (.15 correlation coefficient for year of birth). They work in ethics (.16), social/political philosophy (.16), and 17th-19th century philosophy (.11), avoiding metaphysics (-.2) and the philosophies of mind (-.15) and language (-.14). Their heroes are Kant (.23), Rawls (.14), and, interestingly, Hume (.11). They avoid analytic philosophy even more than the anti-naturalists do (-.17), and aren't fond of Russell (-.11).

5. Externalists: Really, they just like everything that anyone calls 'externalism'. They think the content of our mental lives in general (.66) and perception in particular (.55), and the justification for our beliefs (.64), all depend significantly on the world outside our heads. They also think that you can fully understand a moral imperative without being at all motivated to obey it (.5).

6. Star Trek Haters: This group is less clearly defined than the above ones. The main thing uniting them is that they're thoroughly convinced that teleportation would mean death (.69). Beyond that, Trekophobes tend to be deontologists (.52) who don't switch on trolley dilemmas (.47) and like A theories of time (.41).

Trekophobes are relatively old (-.1) and American (.13 affiliation). They are quite rare in Australia and Asia (-.18 affiliation). They're fairly evenly distributed across philosophical fields, and tend to avoid weirdo intuitions-violating naturalists — Lewis (-.13), Hume (-.12), analytic philosophers generally (-.11).

7. Logical Conventionalists: They two-box on Newcomb's Problem (.58), reject nonclassical logics (.48), and reject epistemic relativism and contextualism (.48). So they love causal decision theory, think all propositions/facts are generally well-behaved (always either true or false and never both or neither), and think there are always facts about which things you know, independent of who's evaluating you. Suspiciously normal.

They're also fond of a wide variety of relatively uncontroversial, middle-of-the-road views most philosophers agree about or treat as 'the default' — political egalitarianism (.33), abstract object realism (.3), and atheism (.27). They tend to think zombies are metaphysically possible (.26) and to reject personal identity reductionism (.26) — which aren't metaphysically innocent or uncontroversial positions, but, again, do seem to be remarkably straightforward and banal approaches to all these problems. Notice that a lot of these positions are intuitive and 'obvious' in isolation, but that they don't converge upon any coherent world-view or consistent methodology. They clearly aren't hard-nosed philosophical conservatives like the Anti-Naturalists, Objectivists, Rationalists, and Trekophobes, but they also clearly aren't upstart radicals like the Externalists (on the analytic side) or the Anti-Realists (on the continental side). They're just kind of, well... obvious.

Conventionalists are the only identified group that are strongly analytic in orientation (.19). They tend to work in epistemology (.16) or philosophy of language (.12), and are rarely found in 17th-19th century (-.12) or continental (-.11) philosophy. They're influenced by notorious two-boxer and modal realist David Lewis (.1), and show an aversion to Hegel (-.12), Aristotle (-.11), and and Wittgenstein (-.1).

An observation: Different philosophers rely on — and fall victim to — substantially different groups of methods and intuitions. A few simple heuristics, like 'don't believe weird things until someone conclusively demonstrates them' and 'believe things that seem to be important metaphysical correlates for basic human institutions' and 'fall in love with any views starting with "ext"', explain a surprising amount of diversity. And there are clear common tendencies to either trust one's own rationality or to distrust it in partial (Externalism) or pathological (Anti-Realism, Anti-Naturalism) ways. But the heuristics don't hang together in a single Philosophical World-View or Way Of Doing Things, or even in two or three such world-views.

There is no large, coherent, consolidated group that's particularly attractive to LWers across the board, but philosophers seem to fall short of LW expectations for some quite distinct reasons. So attempting to criticize, persuade, shame, praise, or even speak of or address philosophers as a whole may be a bad idea. I'd expect it to be more productive to target specific 'load-bearing' doctrines on dimensions like the above than to treat the group as a monolith, for many of the same reasons we don't want to treat 'scientists' or 'mathematicians' as monoliths.

Another important result: Something is going seriously wrong with the high-level training and enculturation of professional philosophers. Or fields are just attracting thinkers who are disproportionately bad at critically assessing a number of the basic claims their field is predicated on or exists to assess.

Philosophers working in decision theory are drastically worse at Newcomb than are other philosophers, two-boxing 70.38% of the time where non-specialists two-box 59.07% of the time (normalized after getting rid of 'Other' answers). Philosophers of religion are the most likely to get questions about religion wrong — 79.13% are theists (compared to 13.22% of non-specialists), and they tend strongly toward the Anti-Naturalism dimension. Non-aestheticians think aesthetic value is objective 53.64% of the time; aestheticians think it's objective 73.88% of the time. Working in epistemology tends to make you an internalist, philosophy of science tends to make you a Humean, metaphysics a Platonist, ethics a deontologist. This isn't always the case; but it's genuinely troubling to see non-expertise emerge as a predictor of getting any important question in an academic field right.

EDIT: I've replaced "cluster" talk above with "dimension" talk. I had in mind gjm's "clusters in philosophical idea-space", not distinct groups of philosophers. gjm makes this especially clear:

The claim about these positions being made by the authors of the paper is not, not even a little bit, "most philosophers fall into one of these seven categories". It is "you can generally tell most of what there is to know about a philosopher's opinions if you know how well they fit or don't fit each of these seven categories". Not "philosopher-space is mostly made up of these seven pieces" but "philosopher-space is approximately seven-dimensional".

I'm particularly guilty of promoting this misunderstanding (including in portions of my own brain) by not noting that the dimensions can be flipped to speak of (anti-anti-)naturalists, anti-rationalists, etc. My apologies. As Douglas_Knight notes below, "If there are clusters [of philosophers], PCA might find them, but PCA might tell you something interesting even if there are no clusters. But if there are clusters, the factors that PCA finds won't be the clusters, but the differences between them. [...] Actually, factor analysis pretty much assumes that there aren't clusters. If factor 1 put you in a cluster, that would tell pretty much all there is to say and would pin down your factor 2, but the idea in factor analysis is that your factor 2 is designed to be as free as possible, despite knowing factor 1."

Post will be returning in Main, after a rewrite by the company's writing staff. Citations Galore.

Giles, and some others, have asked questions about donating to one or more of CEA’s sub-organisations. In what follows, I address these questions. I felt it would be clearest for me to mainly cluster questions under general headings, rather than address the specific wording of every question. (Note: thanks to help from Ben Todd on this!)

A couple of clarifications

Centre for Effective Altruism is a legal entity that comprises 4 organisations: Giving What We Can, 80,000 Hours, Effective Animal Activism, and The Life You Can Save. EAA is formally still a sub-project of 80,000 Hours, but should be thought of as separate for accounting purposes and may well become a separate organisation. In the previous blog post I talked about GWWC and 80k only, but because there’s been interest, here I’ll discuss the other two as well, albeit more briefly.

Some numbers follow. These are true as of Nov 20th 2012 (or, rather, are best estimates as of that date), but are very likely to change in the near future. They should therefore be taken as illustrations merely.

Finally, because all the organisations that comprise CEA are young, there are certain policy issues that still have not been decided upon; and some that have been decided upon may change in the near future as we learn. Where possible, I have tried to flag which policies are as yet undecided.

What’s your expenditure?

Below are rough estimates for expenditure from start Q2 2012 (when we first took staff) until end Q4 2013 (our short-term fundraising horizon). All numbers are in thousands.

80k (basic): £118.8 = ~$190

80k (inc. some expansion): £139.1 = ~$220

GWWC (basic): £87.4 = ~$140

GWWC (inc. some expansion): £107.7 = ~$170

LYCS (basic): £47.88 = ~$76

EAA (basic): £32.76 = ~$52

The large majority of our expenditure is on staff. 80k (basic) comprises one full-time staff member from Q2 2012, two staff members working 0.4 time from Q2 2012, and one full-time staff member from Q1 2013. GWWC (basic) comprises one 5/8 staff member from Q2 2012, and two staff members working 0.6 time from Q2 2012. LYCS (basic) comprises one full-time staff member from Q2 2013, and some money earmarked for on-line marketing by one funder. EAA (basic) comprises one full-time staff member from Q1 2013. Across CEA, we typically employ one intern-year for every employee-year.

For both GWWC and 80k, the difference between the ‘basic’ scenario and the ‘expansion’ scenario is that we would hire one additional person from Q3 2013, and employ one additional intern-year for 2013. The ‘expansion’ scenario indicates a cautious limit on our use for more funding; though it certainly seems to us that we could spend money well above that amount, we would need to discuss whether it could be detrimental in the long-run for the organisations to grow that fast. We could very comfortably spend within the ‘expansion’ scenario, and would feel hindered if we were not able to spend up to that amount.

Each employee is paid a starting salary of £18 per annum, significantly below market rates for graduates even within the not-for-profit world, and without accounting for the fact that our employees are significantly more qualified than the average graduate. Interns are unpaid, but are typically given expenses. A significant proportion of our labour is still voluntary.

What’s your income? What’s your shortfall?

All numbers are now in $000s. I'll measure 'shortfall' relative to the 'basic' budget. The fundraising numbers below include both income that we have already received and income that we expect to receive, discounted according to a conservative estimate of its likelihood (50%). I’ll assume that CEA unrestricted money is divided as follows: 0.4 to 80k, 0.4 to GWWC, 0.12 to LYCS and 0.08 to EAA. (This division does not represent a policy about how we divide unrestricted funds. Currently that policy is not yet determined, so I’ve chosen these numbers as illustrative. More on use of unrestricted money in “earmarking and fungibility” below.)

80k raised: 85.1. Shortfall = 190 - 85.1 = ~$105

GWWC raised: 117.9. Shortfall = 140 – 117.9 = ~$22

LYCS raised: 63.4. Shortfall = 76 – 63.4 = ~$12.5

EAA raised: 27.2. Shortfall = 52 – 27.2 = ~$25

For both GWWC and 80k, to get the shortfall for the 'expansion' budget – which represents spending which we could easily accommodate without sacrificing quality of work —  add 30 to the 'shortfall' number.

How would you spend additional money?

As the numbers above suggest, in most cases additional donations would be spent on basic costs — principally, paying staff — over 2013. If GWWC or 80k exceeded their ‘basic’ budget, then additional money would be put towards on their ‘additional’ budget: principally, hiring one new staff member each.  The desired marginal hire for GWWC is a Communications Director.  The desired marginal hire for 80,000 Hours is a Careers Researcher and Adviser.

Where’s that income from?

Our donations come from a variety of sources. Private donors, of varying degrees of wealth, make up the large majority of our income. GWWC has received a grant from one foundation. The majority of donations come from within the effective giving community, though a sizable proportion comes from outside that community and we’re actively pursuing further leads there, including high net worths. If we ever had a significant donation commitment that used up our room for more funding, we’d let other donors know immediately.

Room for More Funding, and a Co-ordination Problem

Suppose that, within the effective giving community, there is $N that people would want to donate to CEA, conditional on CEA having room for more funding. But CEA only had room for $M, where M<N. Every giver thinks, “well, CEA is going to reach its room or more funding anyway. So there’s no reason why I should given.” So no-one gives to CEA. That would be a bad outcome on our part. Alternatively, perhaps every giver thinks, “well, I’ll just give to CEA anyway”. So CEA receives $N, whereas it can only spend $M well, and there is an $N-$M excess. That would be a bad outcome on the part of the giver. So what’s the solution?

One solution to that problem (though I haven't thought about it that much) is as follows: we decide upon a funding limit for each organisation. We say that if we receive donations above that limit (before a designated time), we will donate the excess to the most cost-effective charities. Different givers think that different causes are the most important, so we’ll donate to the different cause areas depending on what proportion of CEA donations would have been given to those causes if they hadn’t been donated to CEA. So if we received 70% of donations that would have been donated to global poverty, we’ll give 70% of the excess to AMF; similarly for animal welfare and x-risk.

The advantages of this are as follows. It safeguards against CEA having less money than it needs because of the co-ordination problem. It ensures that givers can donate to CEA while knowing that the money won’t be spent on CEA above its room for more funding. It’s also what GiveWell does, and insofar as GW appear to us to be a very well-run organisation, it’s worth imitating them.

What are the alternative solutions? Well, we could bank the money and use it the following year. So the excess money donated to CEA is used one year later than the giver might have expected, and we spend less time on fundraising for the following year. Or we could go ‘first come first served’: we keep accepting donations until we hit our RFMF. I think that the latter suggestion is a bad one. The former is potentially good I think, and simple, and I’m open to comments on which solution potential donors think is preferable.

Different Cause Areas

GWWC and LYCS are focused on global poverty, and have no plans to change that. EAA is focused on animal welfare. 80,000 Hours is open to any plausibly high-impact activity. There is currently no organisation within CEA dedicated purely to x-risk mitigation, but, given demand, it’s not unlikely that one will be created in the mid-term future.

“Earmarking” and Fungibility

Some comments mentioned “earmarking”. I think that’s a misleading term in this context. “Earmarking” normally refers to donations that are tied to a specific activity. Whereas, when one donates to GWWC, the donation is not tied to a specific activity. CEA shouldn’t be thought of as an organization over and above the four organisations.

We actively encourage donations that are restricted to one organization only, if you think that one organization is more cost-effective than the others. In order to avoid the fungibility problem, I considered asking only for restricted donations. It seems to me on balance that the costs of this policy outweigh the benefits, but I’m not sure.

My current preferred solution is as follows. Every 6 months, after the reviews of each organization (see next section), the trustees decide how to allocate unrestricted funding. The default they use is that unrestricted funding is allocated in proportion with restricted funding. If this default holds — either exactly or approximately — then fungibility of donations is not an issue. In fact, if the default holds, then fungibility is negative: donating $1 to GWWC would move slightly more than $1 to GWWC, because it would also increase the proportion of CEA unrestricted money that it receives. The trustees deviate from this default if there are compelling reasons for doing so (e.g. a major donor for one organization unexpectedly drops out, rendering basic expenditure uncertain). In the long run (and ex ante), we wouldn’t expect these deviations to favour one organization over another, so, in the long run (and ex ante), fungibiltiy is again not an issue. Moreover, from this arrangement we would expect each organization to benefit in terms of financial stability and from the success of their sister organisations.

Self-Evaluation and Impact Assessment

I’ll describe 80k’s process. GWWC’s is very similar.

Every 6 months, 80k will have to write a report of its progress over the last 6 months, including achievements and failures, how its progress compares to the goals stated 6 months ago, and write concrete, measurable goals for the next 6 months. This report will then be reviewed by two boards. The trustees of CEA: myself, Nick Beckstead, and Toby Ord. And an "Advisory Committee", consisting of 80k supporters (and often donors) who aren't in any way involved with the running of 80k. The Executive Director of 80k (and one or two others) will meet with these boards, and they'll discuss the report. Each board will write a summary of conclusions. All three documents (initial report, and two commentaries from the boards) will be posted on the blog.

Every year (probably in spring or early summer - a quiet time for us), we'll complete a more in-depth impact-evaluation, at least in terms of money moved, person-hours moved, money pledged and person-hours pledged. All-year round, we measure progress with respect to pre-chosen goals. The ED of 80k sends progress reports to the 80k team every week. Currently, because marketing and recruitment are our key priorities, our principal metrics are number of new members per week, % of members who say that they’ve changed their career plans because of 80k, income pledged per member, unique visitors to the website, and number of advising sessions given.

Miscellaneous CEA Questions

Which is more useful, regular donations or lump sums?

Either is good. Most charities prefer regular donations because people are likely to give more that way (they forget about the direct debit). But I'd rather you were giving on the basis of perceived cost-effectiveness, rather than status quo bias! Financial forecasting is really important for us, though, so if it's lump sums, we really appreciate knowing the chance they'll be repeated in future years. And we have a steep discount rate (perhaps 20%?) so we greatly prefer money sooner rather than later.

If you had funds to hire an extra person, do you know how that person would be? How important is it to find talented people to work for you?  Are you trying to find someone from the top 5%? The top 1%?

It's difficult to give a meaningful reply to that — 1% in terms of general ability, or fit for us? I'll answer for 'general ability' (whatever that means).  We're generally selecting only from top universities, which filters out a large majority of the population. As an approximation (but merely a very rough approximation): There are roughly 772,000 18 yr olds in the UK, of which 7000 go to Oxford or Cambridge. We mainly select from those universities (or equivalent standard elsewhere), so that already filters out 99% of the population.

Within such universities, we normally recruit very high-performing graduates — perhaps in the top 10% or 5%. Which would suggest that we're recruiting from the top 0.1% from the population. But, like I say, I'm not sure that that number is that meaningful. I'm not certain why, but we do seem to be able to recruit exceptionally talented people. (Like Niel, who's starting with us from January).

How much personal connection and communication is there between CEA and these orgs?
- THINK
- Global Catastrophic Risk Institute
- Center for Applied Rationality
- Future of Humanity Institute
- GiveWell

Lots. Mark Lee, founder of THINK, came through GWWC. I gave a talk for the Brown THINK chapter the other day, and helped their co-President with plans for the year. We've met a couple of times with Seth Baum. We know Julia and Anna well, and support CFAR. Toby Ord (a trustee of CEA) is a research associate at FHI, and Will participates in FHI events. We're in regular contact with GiveWell. A core CEA volunteer is considering working for them.

I could say much more, but it would get long-winded. We support all the above organisations, and aim to co-ordinate with them all, so that we don't get in each others' way, and can help each other out.

Where do you see the delineation between what CEA does and what other effective altruist orgs do?

We worry a lot about needlessly doubling up on or competing with work done by other effective altruist organisations. Taking our four organisations in turn:

GWWC: Along with LYCS, the only group in the world promoting major individual cost-effective giving. Does charity effectiveness research, but only where we think we can usefully add to what Givewell does.

80k: The only effective altruist organisation doing careers advice. The most broad focused effective altruist organisation except for THINK, but we’re distinct from THINK in that we provide careers advice through web content and one-on-one sessions rather than setting up meet-ups.

EAA: The only animal focused effective altruist organisation (EAA is effectively doing what 80k would have done in this area, except we thought it was useful to give it separate branding.

TLYCS: Similar in aims to GWWC, but lower-bar entry for most members. Planning different outreach routes

What are you planning in the way of financial transparency?

We'll publish an annual financial report, with a breakdown of costs. We’d like to be able to regularly explain room for more funding (etc.) as we do above, but doing so uses considerable time of high-level people within the organization, so we can’t promise that.  In general, we take GiveWell as a model organisation, and will often emulate their practices.

Miscellaneous GWWC Question

You said in your LW post that you have "much more information available" on GWWC's impact.

Yes. If you email me (will [dot] crouch [at] 80000hours.org), you can see the calculations by which we estimated GWWC’s impact.

Miscellaneous 80k Questions

You do a bunch of different but related things - website content, speaker events, career counseling,
- Do you imagine yourself specialising in just one of these in the future?
- Are you at the stage of experimenting to find out which activity is the most effective?
- Is there synergy between them? (e.g. if career advice sessions and website content are both a lot more effective if you're also doing the other one)

We’re creating a new type of organisation, and there’s a great deal we don’t know. We see our priority as testing these different approaches and improving them. Having a basket of methods lets us gain more information, and prevents us from stalling if one method turns out not to work.

For each method, we track a bunch of metrics which ultimately relate to our bottom line: resources shifted to the most effective causes that wouldn’t have been shifted otherwise. We propose tests for ways to improve these metrics. If our hypotheses about what we expect to work are disconfirmed, we change our approach. Otherwise, we move to scale up the method.

Whether we end up specialising, therefore, depends on whether one of the methods ends up being significantly more effective than the rest. And whatever happens, since we’re constantly seeking to improve, I imagine we’ll always be experimenting with new processes.

At the moment, we broadly see the web content and one-on-one advice as our most important services, and we’re expecting to scale them the most (though there’s a lot of flexibility within where we take these). We pursue some lectures, especially in Oxford, since they have high marginal returns, but we don’t currently expect to scale them. We’re exploring some other methods, but have not tested them yet.

I have very little idea about what the 80K community is like or how exactly you invest in it
- in what ways does your team interact with your community, other than one-on-one career advice and hosting speaker events?
- do you invest in members' skills such as critical thinking and the ability to evaluate organisations?
- what other skills and qualities do you want to develop in your members, and how do you plan to go about it?
- to what extent do you think talents and abilities are inherent (or at least beyond your control), and to what extent are they trainable?

Probably one of the most useful things we do is forge links between people in the community who can help each other out. For instance, we’ve introducing people who have successfully navigated applications to finance jobs to others who want to do the same. We brought together a bunch of people interested in the animal cause to set up EAA. We’ve introduced people who are in the same research field. At the minute, this mainly happens via personal introductions, but we’re developing tools to make this easier online.
Besides this, the team interacts with the community via the members’ googlegroup and our online discussion forum, 80000hours.org/discussion.

Our current focus in our providing our members with really useful information about which career they should enter in order to have the most impact. Our main way of improving their skills is by introducing them to mentors. We also sometimes coach people through tough career stages in our one-on-one advice (e.g. we recently helped get one of our interns a Marshall scholarship). Longer term, we might switch to have a greater focus on self-improvement, but that niche has a lot more competition (e.g. if you care about becoming more rational, go to Less Wrong).

Is 80K planning activity in any new physical locations?
- If so, where?
- If there's already a THINK community in that location, what do you imagine the relationship between THINK and 80K looking like?

We encourage any 80k members who’d like to start an 80k meet-up to do so, and we’ll happily give them advice and support; but we don’t plan to invest significant staff time in setting up new physical locations. This is because our current focus is on our web content and one-on-one advice. The exception is that we’d like to intensify existing involvement in Oxford, Yale and Princeton, which we see as test grounds and already have some infrastructure.

We could easily change our minds, however, and we’re pretty interested in the idea of doing a lecture tour (which we could support with the web content and one-on-one advice).  We wouldn’t set up an 80k presence in a new location, as opposed to sending people to THINK, unless we thought that there was good reason that having two organisations in the area would be more effective.

There's apparently a lot of interest in x-risk among 80K members. Do you know why this is?

I think that the 34% is higher than the percentage of x-risk concerned members in the long-run (which I'd guess will end up at about 10%), due to initial selection effects (we got quite a few members from the x-risk network, and others who would have been GWWC members were if not for the fact that they were principally concerned about x-risk). However, there does seem to be a strong positive correlation between how dedicated members are, and whether they are concerned by x-risk. And we haven’t yet really discussed x-risk as a cause area. So it’s difficult to say what proportion of resources we generate will be x-risk focused.

A number of people are convinced by x-risk but just don't think that there currently exists a good enough giving opportunity. So the proportion would increase considerably if a really clear x-risk giving opportunity arose (e.g. if GW ever recommended an x-risk org).

Do you know of any other organisations that do anything similar to what you do? (other than ones I've already mentioned). In particular any groups that give career advice to philanthropists.

As far as we can tell, there’s no-one else providing careers advice focused on how you can make a difference. All that exists is informal advice about impact given by friends and within other effective altruist communities (e.g. LW, GWWC).
Making a difference aside, it seems like the average quality of careers advice in general is pretty low, and rarely evidence-based or aware of decision making biases.

From several perspectives, we think we’re in a very interesting market niche. As far as philanthropy goes, there are some groups focused on fostering it e.g. http://youngphilanthropy.org.uk/, but they tend to have little focus on effectiveness.

Miscellaneous EAA Questions

What are the plans for EAA? When will it be spun off? Is there much interest in it from new members of 80k? Is anybody other than Eitan Fischer (who's in school) working on it at the moment?

EAA is taking on a full-time Executive Director some time between Jan and Sept. The initial priorities will be scaling up the charity effectiveness research, taking it to major philanthropists and fundraising. If EAA achieves enough scale, we’ll promote it to a full member of CEA.

Around 10-20% of our new members are interested in the animal cause (and we haven’t promoted it much directly, beyond a couple of blog posts). They tend to make use of EAA. Eitan has a small team of volunteers and advisers helping him out part-time. There’s probably about 4-8 people involved in some capacity at most times. This should increase significantly once we recruit an Executive Director.

I sometimes go around saying that the fundamental question of rationality is Why do you believe what you believe?

-- Eliezer in Quantum Non-Realism

I was much impressed when they finally came out with a PC version of DragonBox, and I got around to testing it on some children I knew. Two kids, one of them four and the other eight years old, ended up blazing through several levels of solving first-degree equations while having a lot of fun doing so, even though they didn't know what it was that they were doing. That made me think that there has to be some way of making a computer game that would similarly teach rationality skills at the 5-second level. Some game where you would actually be forced to learn useful skills if you wanted to make progress.

After playing around with some ideas, I hit upon the notion of making a game centered around the Fundamental Question. I'm not sure whether this can be made to work, but it seems to have promise. The basic idea: you are required to figure out the solution to various mysteries by collecting various kinds of evidence. Some of the sources of evidence will be more reliable than others. In order to hit upon the correct solution, you need to consider where each piece of evidence came from, and whether you can rely on it.

Gameplay example

Now, let's go into a little more detail. Let's suppose that the game has a character called Bob. Bob tells you that tomorrow, eight o'clock, there will be an assassination attempt on Market Square. The fact that Bob has told you this is evidence for the claim being true, so the game automatically records the fact that you have such a piece of evidence, and that it came from Bob.

(Click on the pictures in case you don't see them properly.)

But how does Bob know that? You ask, and it turns out that Alice told him. So next, you go and ask Alice. Alice is confused and says that she never said anything about any assassination attempt: she just said that something big is going to be happen at the Market Square at that time, she heard it from the Mayor. The game records two new pieces of evidence: Alice's claim of something big happening at the Market Square tomorrow (which she heard from the Mayor), and her story of what she actually told Bob. Guess that Bob isn't a very reliable source of evidence: he has a tendency to come up with fancy invented details.

Or is he? After all, your sole knowledge about Bob being unreliable is that Alice claims she never said what Bob says she said. But maybe Alice has a grudge against Bob, and is intentionally out to make everyone disbelieve him. Maybe it's Alice who's unreliable. The evidence that you have is compatible with both hypotheses. At this point, you don't have enough information to decide between them, but the game lets you experiment with setting either of them as "true" and seeing the implications of this on your belief network. Or maybe they're both true - Bob is generally unreliable, and Alice is out to discredit him. That's another possibility that you might want to consider. In any case, the claim that there will be an assassination tomorrow isn't looking very likely at the moment.

Actually, having the possibility for somebody lying should probably be a pretty late-game thing, as it makes your belief network a lot more complicated, and I'm not sure whether this thing should display numerical probabilities at all. Instead of having to juggle the hypotheses of "Alice lied" and "Bob exaggerates things", the game should probably just record the fact that "Bob exaggerates things". But I spent a bunch of time making these pictures, and they do illustrate some of the general principles involved, so I'll just use them for now.

Game basics

So, to repeat the basic premise of the game, in slightly more words this time around: your task is to figure out something, and in order to do so, you need to collect different pieces of evidence. As you do so, the game generates a belief network showing the origin and history of the various pieces of evidence that you've gathered. That much is done automatically. But often, the evidence that you've gathered is compatible with many different hypotheses. In those situations, you can experiment with different ways of various hypotheses being true or false, and the game will automatically propagate the consequences of that hypothetical through your belief network, helping you decide what angle you should explore next.

Of course, people don't always remember the source of their knowledge, or they might just appeal to personal experiences. Or they might lie about the sources, though that will only happen at the more advanced levels.

As you proceed in the game, you will also be given access to more advanced tools that you can use for making hypothetical manipulations to the belief network. For example, it may happen that many different characters say that armies of vampire bats tend to move about at full moon. Since you hear that information from many different sources, it seems reliable. But then you find out that they all heard it from a nature documentary on TV that aired a few weeks back. This is reflected in your belief graph, as the game modifies it to show that all of those supposedly independent sources can actually be tracked back to a single one. That considerably reduces the reliability of the information.

But maybe you were already suspecting that the sources might not be independent? In that case, it would have been nice if the belief graph interface would let you postulate this beforehand, and see how big of an effect it would make on the plausibility of the different hypotheses if they were in fact reliant on each other. Once your character learns the right skills, it becomes possible to also add new hypothetical connections to the belief graph, and see how this would influence your beliefs. That will further help you decide what possibilities to explore and verify.

Because you can't explore every possible eventuality. There's a time limit: after a certain amount of moves, a bomb will go off, the aliens will invade, or whatever.

The various characters are also more nuanced than just "reliable" or "not reliable". As you collect information about the various characters, you'll figure out their mindware, motivations, and biases. Somebody might be really reliable most of the time, but have strong biases when it comes to politics, for example. Others are out to defame others, or invent fancy details to all the stories. If you talk to somebody you don't have any knowledge about yet, you can set a prior on the extent that you rely on their information, based on your experiences with other people.

You also have another source of evidence: your own intuitions and experience. As you get into various situations, a source of evidence that's labeled simply "your brain" will provide various gut feelings and impressions about things. The claim that Alice presented doesn't seem to make sense. Bob feels reliable. You could persuade Carol to help you if you just said this one thing. But in what situations, and for what things, can you rely on your own brain? What are your own biases and problems? If you have a strong sense of having heard something at some point, but can't remember where it was, are you any more reliable than anyone else who can't remember the source of their information? You'll need to figure all of that out.

As the game progresses to higher levels, your own efforts will prove insufficient for analyzing all the necessary information. You'll have to recruit a group of reliable allies, who you can trust to analyze some of the information on their own and report the results to you accurately. Of course, in order to make better decisions, they'll need you to tell them your conclusions as well. Be sure not to report as true things that you aren't really sure about, or they will end up drawing the wrong conclusions and focusing on the wrong possibilities. But you do need to condense your report somewhat: you can't just communicate your entire belief network to them.

Hopefully, all of this should lead to player learning on a gut level things like:

• Consider the origin of your knowledge: Obvious.
• Visualizing degrees of uncertainty: In addition to giving you a numerical estimate about the probability of something, the game also color-codes the various probabilities and shows the amount of probability mass associated with your various beliefs.
• Considering whether different sources really are independent: Some sources which seem independent won't actually be that, and some which seem dependent on each other won't be.
• Value of information: Given all the evidence you have so far, if you found out X, exactly how much would it change your currently existing beliefs? You can test this and find out, and then decide whether it's worth finding out.
• Seek disconfirmation: A lot of things that seem true really aren't, and acting on flawed information can cost you.
• Prefer simpler theories: Complex, detailed hypotheses are more likely to be wrong in this game as well.
• Common biases: Ideally, the list of biases that various characters have is derived from existing psychological research on the topic. Some biases are really common, others are more rare.
• Epistemic hygiene: Pass off wrong information to your allies, and it'll cost you.
• Seek to update your beliefs: The game will automatically update your belief network... to some extent. But it's still possible for you to assign mutually exclusive events probabilities that sum to more than 1, or otherwise have conflicting or incoherent beliefs. The game will mark these with a warning sign, and it's up to you to decide whether this particular inconsistency needs to be resolved or not.
• Etc etc.
  

Design considerations

It's not enough for the game to be educational: if somebody downloads the game because it teaches rationality skills, that's great, but we want people to also play it because it's fun. Some principles that help ensure that, as well as its general utility as an educational aid, include:

• Provide both short- and medium-term feedback: Ideally, there should be plenty of hints for how to find out the truth about something by investigating just one more thing: then the player can find out whether your guess was correct. It's no fun if the player has to work through fifty decisions before finding out whether they made the right move: they should get constant immediate feedback. At the same time, the player's decisions should be building up to a larger goal, with uncertainty about the overall goal keeping them interested.
• Don't overwhelm the player: In a game like this, it would be easy to throw a million contradictory pieces of evidence at the player, forcing them to go through countless of sources of evidence and possible interactions and have no clue of what they should be doing. But the game should be manageable. Even if it looks like there is a huge messy network of countless pieces of contradictory evidence, it should be possible to find the connections which reveal the network to be relatively simple after all. (This is not strictly realistic, but necessary for making the game playable.)
• Introduce new gameplay concepts gradually: Closely related to the previous item. Don't start out with making the player deal with every single gameplay concept at once. Instead, start them out in a trusted and safe environment where everyone is basically reliable, and then begin gradually introducing new things that they need to take into account.
• No tedium: A game is a series of interesting decisions. The game should never force the player to do anything uninteresting or tedious. Did Alice tell Bob something? No need to write that down, the game keeps automatic track of it. From the evidence that has been gathered so far, is it completely obvious what hypothesis is going to be right? Let the player mark that as something that will be taken for granted and move on.
• No glued-on tasks: A sign of a bad educational game is that the educational component is glued on to the game (or vice versa). Answer this exam question correctly, and you'll get to play a fun action level! There should be none of that - the educational component should be an indistinguishable part of the game play.
• Achievement, not fake achievement: Related to the previous point. It would be easy to make a game that wore the attire of rationality, and which used concepts like "probability theory", and then when your character leveled up he would get better probability attacks or whatever. And you'd feel great about your character learning cool stuff, while you yourself learned nothing. The game must genuinely require the player to actually learn new skills in order to get further.
• Emotionally compelling: The game should not be just an abstract intellectual exercise, but have an emotionally compelling story as well. Your choices should feel like they matter, and characters should be in risk of dying if you make the wrong decisions.
• Teach true things: Hopefully, the players should take the things that they've learned from the game and apply them to their daily lives. That means that we have a responsibility not to teach them things which aren't actually true.
• Replayable: Practice makes perfect. At least part of the game world needs to be randomly generated, so that the game can be replayed without a risk of it becoming boring because the player has memorized the whole belief network.

What next?

What you've just read is a very high-level design, and a quite incomplete one at that: I've spoken on the need to have "an emotionally compelling story", but said nothing about the story or the setting. This should probably be something like a spy or detective story, because that's thematically appropriate for a game which is about managing information; and it might be best to have it in a fantasy setting, so that you can question the widely-accepted truths of that setting without needing to get on anyone's toes by questioning widely-accepted truths of our society.

But there's still a lot of work that remains to be done with regard to things like what exactly does the belief network look like, what kinds of evidence can there be, how does one make all of this actually be fun, and so on. I mentioned the need to have both short- and medium-term feedback, but I'm not sure of how that could be achieved, or whether this design lets you achieve it at all. And I don't even know whether the game should show explicit probabilities.

And having a design isn't enough: the whole thing needs to be implemented as well, preferably while it's still being designed in order to take advantage of agile development techniques. Make a prototype, find some unsuspecting testers, spring it on them, revise. And then there are the graphics and music, things for which I have no competence for working on.

I'll probably be working on this in my spare time - I've been playing with the idea of going to the field of educational games at some point, and want the design and programming experience. If anyone feels like they could and would want to contribute to the project, let me know.

EDIT: Great to see that there's interest! I've created a mailing list for discussing the game. It's probably easiest to have the initial discussion here, and then shift the discussion to the list.

Over the last few months I've started blogging about effective altruism more broadly, rather than focusing on AI risk. I'm still focusing on abstract considerations and methodological issues, but I hope it is of interest to others here. Going forward I intend to cross-post more often to LW, but I thought I would post the backlog here anyway. With luck, I'll also have the opportunity to post more than bi-weekly.

I welcome thoughts, criticisms, etc.

Eliezer proposed in a comment:

>More difficult version of AI-Box Experiment: Instead of having up to 2 hours, you can lose at any time if the other player types AI DESTROYED. The Gatekeeper player has told their friends that they will type this as soon as the Experiment starts. You can type up to one sentence in your IRC queue and hit return immediately, the other player cannot type anything before the game starts (so you can show at least one sentence up to IRC character limits before they can type AI DESTROYED). Do you think you can win?

This spawned a flurry of ideas on what the AI might say. I think there's a lot more ideas to be mined in that line of thought, and the discussion merits its own thread.

So, give your suggestion - what might an AI might say to save or free itself?

(The AI-box experiment is explained here)

EDIT: one caveat to the discussion: it should go without saying, but you probably shouldn't come out of this thinking, "Well, if we can just avoid X, Y, and Z, we're golden!" This should hopefully be a fun way to get us thinking about the broader issue of superinteligent AI in general. (Credit goes to Elizer, RichardKennaway, and others for the caveat)

AidGrade is a new charity evaluator that looks to be comparable to GiveWell. Their primary difference is that they *only* focus on how charities compare along particular measured outcomes (such as school attendance, birthrate, chance of opening a business, malaria), without making any effort to compare between types of charities. (This includes interesting results like "Conditional Cash Transfers and Deworming are better at improving attendance rates than scholarships")

GiveWell also does this, but designs their site to direct people towards their top charities. This is better for people with don't have the time to do the (fairly complex) work of comparing charities across domains, but AidGrade aims to be better for people that just want the raw data and the ability to form their own conclusions.

I haven't looked it enough to compare the quality of the two organizations' work, but I'm glad we finally have another organization, to encourage some competition and dialog about different approaches.

This is a fun page to play around with to get a feel for what they do:
http://www.aidgrade.org/compare-programs-by-outcome

And this is a blog post outlining their differences with GiveWell:
http://www.aidgrade.org/uncategorized/some-friendly-concerns-with-givewell