steven0461 comments on Preface to a Proposal for a New Mode of Inquiry - Less Wrong

4 Post author: Daniel_Burfoot 17 May 2010 02:11AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (83)

You are viewing a single comment's thread. Show more comments above.

Comment author: rhollerith_dot_com 19 May 2010 12:47:46AM *  3 points [-]

In discussions on Overcoming Bias during the last 3 years, before and after LW spun off of Overcoming Bias, most people voicing opinions backed by actual reasoning voiced opinions that assigned a higher probability to a hard take-off given that a self-improving AGI is created than Robin.

In the spirit of impartial search for the truth, I will note that rwallace on LW advocates not worrying about unFriendly AI, but I think he has invested years becoming an AGI researcher. Katja Grace is another who thinks hard take-off very unlikely and has actual reasoning on her blog to that effect. She has not invested any time becoming an AGI researcher and has lived for a time at Benton Street as a Visiting Fellow and in the Washington, D.C., area where she traveled with the express purpose of learning from Robin Hanson.

All the full-time employees and volunteers of SIAI that I know of assign much more probability to hard take-off (given AGI) than Robin does. At a workshop following last year's Singularity Summit, every attendee expressed the wish that brain emulation would arrive before AGI. I get the definite impression that those wishes stems mainly from fears of hard takeoff, and not from optimism about brain emulation per se. In the spirit of impartial search for truth, I note that SIAI employees and volunteers probably chose the attendee list of this workshop.

Comment author: steven0461 19 May 2010 01:19:46AM 2 points [-]

Also, people who believe hard takeoff is plausible are more likely to want to work with SIAI, and people at SIAI will probably have heard the pro-hard-takeoff arguments more than the anti-hard-takeoff arguments. That said, <1% is as far as I can tell a clear outlier among those who have thought seriously about the issue.

Comment author: Will_Newsome 20 May 2010 06:52:51AM 3 points [-]

When Robin visited Benton house and the 1% figure was brought up, he was skeptical that he had ever made such a claim. Do you know where that estimate came up (on OB or wherever)? I'm worried about ascribing incorrect probability estimates to people who are fully able to give new ones if we asked.

Comment author: arundelo 20 May 2010 07:48:05AM *  0 points [-]

Off-topic question: Is Benton house the same as the SIAI house? (I see that it is in the Bay Area.) Edit: Thanks Nick and Kevin!

Comment author: Kevin 20 May 2010 08:01:38AM 2 points [-]

The people living there seem to call it Benton house or Benton but I try to avoid calling it that to most people because it is clearly confusing. It'll be even more confusing if the SIAI house moves from Benton Street...

Comment author: Nick_Tarleton 20 May 2010 07:53:53AM 2 points [-]

Yes.