ciphergoth comments on Rationality Quotes September 2013 - Less Wrong

5 Post author: Vaniver 04 September 2013 05:02AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (456)

You are viewing a single comment's thread. Show more comments above.

Comment author: lukeprog 14 September 2013 05:58:49PM *  8 points [-]

I have a sense you misunderstood me. I picture this parallel world where legitimate, rational inferences about the AI risk exist, and where this risk is worth working at in 2013 and stands out among the other risks, as well as any other pre-requisites for making MIRI worthwhile hold. And in this imaginary world, I expect massively larger support than "Steven Hawkins hooked up with FHI" or what ever you are outlining here.

Right, this just goes back to the same disagreement in our models I was trying to address earlier by making predictions. Let me try something else, then. Here are some relevant parts of my model:

  1. I expect most highly credentialed people to not be EAs in the first place.
  2. I expect most highly credentialed people to not be familiar with the arguments for caring about the far future.
  3. I expect most highly credential people to be mostly just aware of risks they happen to have heard about (e.g. climate change, asteroids, nuclear war), rather than attempting a systematic review of risks (e.g. by reading the GCR volume).
  4. I expect most highly credentialed people to respond fairly well when actuarial risk is easily calculated (e.g. asteroid risk), and not-so-well when it's more difficult to calculate (e.g. many insurance companies went bankrupt after 9/11).
  5. I expect most highly credentialed people to have spent little time on explicit calibration training.
  6. I expect most highly credentialed people to not systematically practice debiasing like some people practice piano.
  7. I expect most highly credentialed people to know very little about AI, and very little about AI risk.
  8. I expect that in general, even those highly credentialed people who intuitively think AI risk is a big deal will not even contact the people who think about AI risk for a living in order to ask about their views and their reasons for them, due to basic VoI failure.
  9. I expect most highly credentialed people to have fairly reasonable views within their own field, but to often have crazy views "outside the laboratory."
  10. I expect most highly credentialed people to not have a good understanding of Bayesian epistemology.
  11. I expect most highly credentialed people to continue working on, and caring about, whatever their career has been up to that point, rather than suddenly switching career paths on the basis of new information and an EV calculation.
  12. I expect most highly credentialed people to not understand lots of pieces of "black swan epistemology" like this one and this one.
  13. etc.
Comment author: ciphergoth 15 September 2013 08:43:02AM 9 points [-]

Luke, why are you arguing with Dmytry?