CarlShulman comments on How does MIRI Know it Has a Medium Probability of Success? - Less Wrong

19 Post author: peter_hurford 01 August 2013 11:42AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (137)

Comments (137)

You are viewing a single comment's thread. Show more comments above.

Comment author: CarlShulman 04 August 2013 08:04:35PM *  8 points [-]

If the year was 1960, which would you rather have?

  • 10 smart people trying to build FAI for 20 years, 1960-1980
  • A billion dollars, a large supporting movement, prediction markets and science courts that make the state of the evidence on AI transparent, and teams working on FAI, brain emulations, cognitive enhancement, and more but starting in 1990 (in expectation closer to AI)

At any given time there are many problems where solutions are very important, but the time isn't yet right to act on them, rather than on the capabilities to act on them, and also to deal with the individually unexpected problems that come along so regularly. Investment-driven and movement-building-driven discount rates are relevant even for existential risk.

GiveWell has grown in influence much faster than the x-risk community while working on global health, and are now in the process of investigating and pivoting towards higher leverage causes, with global catastrophic risk among the top three under consideration.

Comment author: Eliezer_Yudkowsky 04 August 2013 08:12:43PM 3 points [-]

I'd rather have both, hence diverting some marginal resources to CFAR until it was launched, then switching back to MIRI. Is there a third thing that MIRI should divert marginal resources to right now?

Comment author: CarlShulman 04 August 2013 08:31:44PM *  13 points [-]

I have just spent a month in England interacting extensively with the EA movement here (maybe your impressions from the California EA summit differ, I'd be curious to hear). Donors interested in the far future are also considering donations to the following (all of these are from talks with actual people making concrete short-term choices; in addition to donations, people are also considering career choices post-college):

  • 80,000 Hours, CEA and other movement building and capacity-increasing organizations (including CFAR), which also increase non-charity options (e.g. 80k helping people going into scientific funding agencies and political careers where they will be in a position to affect research and policy reactions to technologies relevant to x-risk and other trajectory changes)
  • AMF/GiveWell charities to keep GiveWell and the EA movement growing while actors like GiveWell, Paul Christiano, Nick Beckstead and others at FHI, investigate the intervention options and cause prioritization, followed by organization-by-organization analysis of the GiveWell variety, laying the groundwork for massive support for the top far future charities and organizations identified by said processes
  • Finding ways to fund such evaluation with RFMF, e.g. by paying for FHI or CEA hires to work on them
  • The FHI's other work
  • A donor-advised fund investing the returns until such evaluations or more promising opportunities present themselves or are elicited by the fund (possibilities like Drexler's nanotech panel, extensions of the DAGGRE methods, a Bayesian aggregation algorithm that greatly improves extraction of scientific expert opinion or science courts that could mobilize much more talent and resources to neglected problems with good cases, some key steps in biotech enhancement)

That's why Peter Hurford posted the OP, because he's an EA considering all these options, and wants to compare them to MIRI.

Comment author: Eliezer_Yudkowsky 04 August 2013 08:45:54PM 13 points [-]

That is a sort of discussion my brain puts in a completely different category. Peter and Carl, please always give me a concrete alternative policy option that (allegedly) depends on a debate, if such is available; my brain is then far less likely to label the conversation "annoying useless meta objections that I want to just get over with as fast as possible".

Can we have a new top-level comment on this?

Comment author: CarlShulman 04 August 2013 10:39:45PM *  5 points [-]

I edited my top-level comment to include the list and explanation.

Comment author: Rain 04 August 2013 11:58:40PM *  0 points [-]

AMF/GiveWell charities to keep GiveWell and the EA movement growing while actors like GiveWell, Paul Christiano, Nick Beckstead and others at FHI, investigate the intervention options and cause prioritization, followed by organization-by-organization analysis of the GiveWell variety, laying the groundwork for massive support for the top far future charities and organizations identified by said processes

Cool, if MIRI keeps going, they might be able to show FAI as top focus with adequate evidence by the time all of this comes together.

Comment author: lukeprog 07 August 2013 04:19:35AM 0 points [-]

Well, in collaboration with FHI. As soon as Bostrom's Superintelligence is released, we'll probably be building on and around that to make whatever cases we think are reasonable to make.

Powered by Reddit Powered by Reddit