steven0461 comments on The Importance of Self-Doubt - Less Wrong

23 Post author: multifoliaterose 19 August 2010 10:47PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (726)

You are viewing a single comment's thread. Show more comments above.

Comment author: steven0461 20 August 2010 09:02:03PM *  15 points [-]

One could offer many crude back-of-envelope probability calculations. Here's one: let's say there's

  • a 10% chance AGI is easy enough for the world to do in the next few decades
  • a 1% chance that if the world can do it, a team of supergeniuses can do the Friendly kind first
  • an independent 10% chance Eliezer succeeds at putting together such a team of supergeniuses

That seems conservative to me and implies at least a 1 in 10^4 chance. Obviously there's lots of room for quibbling here, but it's hard for me to see how such quibbling could account for five orders of magnitude. And even if post-quibbling you think you have a better model that does imply 1 in 10^9, you only need to put little probability mass on my model or models like it for them to dominate the calculation. (E.g., a 9 in 10 chance of a 1 in 10^9 chance plus a 1 in 10 chance of a 1 in 10^4 chance is close to a 1 in 10^5 chance.)

Comment author: multifoliaterose 20 August 2010 09:58:40PM *  1 point [-]

I don't find these remarks compelling. I feel similar remarks could be used to justify nearly anything. Of course, I owe you an explanation. One will follow later on.