= 27d567bc2d48d9f40e9ccc20ea370b15)
XiXiDu comments on [Link] John Baez interviews Eliezer - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (22)
...
Can this be interpreted the way that Eliezer Yudkowsky believes that he himself, or the SIAI, will not only define friendliness but actually implement it and run a fooming AI to take over the universe? If they really believe that and if it is likely that they can succeed, I still think that even given a very low probability of them being dishonest one should seriously consider how it can be guaranteed that the AI they run is actually friendly. Let me ask you people who believe that the SIAI can succeed, are you not worried at all about unfriendly humans? You just trust their words? That's really weird. If I don't misunderstand what he is saying in those two quotes above, or if he isn't joking, he's actually saying that he'll run a fooming AI.
If you want to offer a concrete proposal for verifying the trustworthiness of nine people in a basement, offer it. Otherwise you're just giving people an excuse to get lost in thought and implement the do-nothing option instead of implementing the best time-sensitive policy proposal offered so far.
He has
I'm not sure I would personally endorse those possibilities, but let it not be said that he complains without offering solutions.
I came up with the above in about 1 minute. You shouldn't even have to ask me how one could verify the trustworthiness of a charity. There are many more obvious ways to approach that problem.
Those sound like good ideas (except the first one), but they aren't ideas for allaying your fears that SIAI will make an evil AI (except the last one). They are ideas for allaying your fears that SIAI won't put your donation to good use. (Except the last one.)
Yes - they'd show SIAI is doing something, but not that it's doing the right thing. And a 99% competent SIAI could well be worse than a 0% competent one – if they create a fooming UFAI a few years earlier.
It seems hard to think of anything that would verify that the nine are doing the right thing without risking AGI knowledge leaking out - I'd much sooner take my chances with a bunch of dudes in a basement who at least know there's a problem then an IBM team who just want moar awesum.
If Friendliness turns out to be largely independent of the AGI bit I suppose it could be usefully published - both for feedback, and to raise awareness, and LW etc. could critique it.
The realistic outcomes for humanity are uFAI foom, FAI foom, or extinction by some other means. Soon doesn't matter all that much; the only significant question is probability of an eventual Friendly foom. Those "few years earlier" only matter if someone else would have run a Friendly AGI in those few intervening years.
EDITED TO ADD: None of this changes the substance of your article, but just to pick a few nits:
"Foom" refers to a scenario in which we reach superintelligence rapidly enough to take humanity by surprise. That isn't certain - it's imaginable that we could have, say, several years of moderately superhuman intelligence.
Also, while these may be the long term realistic outcomes, in the short term another possible outcome is global catastrophe short of extinction, which would slow things down some.
I don't think any of that changes the substance of my argument.
Sorry, should have been clearer that I was just nitpicking. Will edit.
They are ideas for allaying fears that SIAI is incompetent or worse. Which, since it is devoted to building an AI, would tend to allay fears that it is building an evil one.
Basically incompetent organizations that try to build AI just won't do anything.
I'm especially interested in this. I'm open to the idea that SIAI is the maximally useful charity, but since I don't know why they need the money I'm currently giving it to Village Reach.
I don't know whether there's any way to absolutely prove that SIAI will get it right (though I hope that if they come up with a proof of Friendliness they make it public), but I trust them more than their most likely competitors which I think would be governments.