Nisan comments on [Link] John Baez interviews Eliezer - Less Wrong

15 Post author: Morendil 07 March 2011 07:49AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (22)

Comments (22)

You are viewing a single comment's thread. Show more comments above.

Comment author: XiXiDu 07 March 2011 05:14:17PM *  12 points [-]

If you want to offer a concrete proposal for verifying the trustworthiness of nine people in a basement, offer it.

  • Pay independent experts to peer-review your work.
  • Make the finances of the SIAI easily accessible.
  • Openly explain why and for what you currently need more money.
  • Publish progress reports for people to assess how close you are to run a fooming AI.
  • Publish a roadmap, set certain goals and openly pronounce success or failure.
  • Devise a plan that allows the examination by independent experts of a possible seed AI before you run it.

I came up with the above in about 1 minute. You shouldn't even have to ask me how one could verify the trustworthiness of a charity. There are many more obvious ways to approach that problem.

Comment author: Nisan 07 March 2011 06:21:45PM 12 points [-]

Those sound like good ideas (except the first one), but they aren't ideas for allaying your fears that SIAI will make an evil AI (except the last one). They are ideas for allaying your fears that SIAI won't put your donation to good use. (Except the last one.)

Comment author: Larks 07 March 2011 08:26:04PM 9 points [-]

Yes - they'd show SIAI is doing something, but not that it's doing the right thing. And a 99% competent SIAI could well be worse than a 0% competent one – if they create a fooming UFAI a few years earlier.

It seems hard to think of anything that would verify that the nine are doing the right thing without risking AGI knowledge leaking out - I'd much sooner take my chances with a bunch of dudes in a basement who at least know there's a problem then an IBM team who just want moar awesum.

If Friendliness turns out to be largely independent of the AGI bit I suppose it could be usefully published - both for feedback, and to raise awareness, and LW etc. could critique it.

Comment author: Pavitra 08 March 2011 03:50:53AM 3 points [-]

The realistic outcomes for humanity are uFAI foom, FAI foom, or extinction by some other means. Soon doesn't matter all that much; the only significant question is probability of an eventual Friendly foom. Those "few years earlier" only matter if someone else would have run a Friendly AGI in those few intervening years.

Comment author: ciphergoth 08 March 2011 10:44:01AM *  1 point [-]

EDITED TO ADD: None of this changes the substance of your article, but just to pick a few nits:

"Foom" refers to a scenario in which we reach superintelligence rapidly enough to take humanity by surprise. That isn't certain - it's imaginable that we could have, say, several years of moderately superhuman intelligence.

Also, while these may be the long term realistic outcomes, in the short term another possible outcome is global catastrophe short of extinction, which would slow things down some.

Comment author: Pavitra 08 March 2011 06:34:43PM 1 point [-]

I don't think any of that changes the substance of my argument.

Comment author: ciphergoth 09 March 2011 12:05:49PM 2 points [-]

Sorry, should have been clearer that I was just nitpicking. Will edit.

Comment author: nerzhin 07 March 2011 07:13:03PM 2 points [-]

They are ideas for allaying fears that SIAI is incompetent or worse. Which, since it is devoted to building an AI, would tend to allay fears that it is building an evil one.

Comment author: Eliezer_Yudkowsky 07 March 2011 08:17:10PM 8 points [-]

Basically incompetent organizations that try to build AI just won't do anything.

Powered by Reddit Powered by Reddit