= 27d567bc2d48d9f40e9ccc20ea370b15)
steven0461 comments on The Importance of Self-Doubt - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (726)
I don't think there's any point doing armchair diagnoses and accusing people of delusions of grandeur. I wouldn't go so far as to claim that Eliezer needs more self-doubt, in a psychological sense. That's an awfully personal statement to make publicly. It's not self-confidence I'm worried about, it's insularity.
Here's the thing. The whole SIAI project is not publicly affiliated with (as far as I've heard) other, more mainstream institutions with relevant expertise. Universities, government agencies, corporations. We don't have guest posts from Dr. X or Think Tank Fellow Y. The ideas related to friendly AI and existential risk have not been shopped to academia or evaluated by scientists in the usual way. So they're not being tested stringently enough.
It's speculative. It feels fuzzy to me -- I'm not an expert in AI, but I have some education in math, and things feel fuzzy around here.
If you want to claim you're working on a project that may save the world, fine. But there's got to be more to show for it, sooner or later, than speculative essays. At the very least, people worried about unfriendly AI will have to gather data and come up with some kind of statistical study that gives evidence of a threat! Look at climate science. For all the foibles and challenges of the climate change movement, those people actually gather data, create prediction models, predict the results of mitigating policies -- it works more or less like science.
If I'm completely off base here and SIAI is going to get to the science soon, I apologize, and I'll shut up about this for a while.
But look. All this advice about the "sin of underconfidence" is all very well (and actually I've taken it to heart somewhat.) But if you're going to go test your abilities, then test them. Against skeptics. Against people who'll look at you like you're a rotten fish if you don't have a graduate degree. Get something about FAI peer-reviewed or published by a reputable press. Show us something.
Sorry to be so blunt. It's just that I want this to be something. And I have my doubts because there's doesn't seem to be enough in this floating world in the way of unmistakable, concrete achievement.
According to the about page, LW is brought to you by the Future of Humanity Institute at Oxford University. Does this count? Many Dr. Xes have spoken at the Singularity Summits.
It's not clear how one would use past data to give evidence for or against a UFAI threat in any straightforward way. There's various kinds of indirect evidence that could be presented, and SIAI has indeed been trying more in the last year or two to publish articles and give conference talks presenting such evidence.
Points that SIAI would do better if it had better PR, had more transparency, published more in the scientific literature, etc., are all well-taken, but these things use limited resources, which to me makes it sound strange to use them as arguments to direct funding elsewhere.
My post was by way of explaining why some people (including myself) doubt the claims of SIAI. People doubt claims when, compared to other claims, they're not justified as rigorously, or haven't met certain public standards. Why do I agree with the main post that Eliezer isn't justified in his opinion of his own importance (and SIAI's importance)? Because there isn't (yet) a lot beyond speculation here.
I understand about limited resources. If I were trying to run a foundation like SIAI, I might do exactly what it's doing, at first, and then try to get the academic credentials. But as an outside person, trying to determine: is this worth my time? Is this worth further study? Is this a field I could work in? Is this worth my giving away part of my (currently puny) income in donations? I'm likely to hold off until I see something stronger.
And I'm likely to be turned off by statements with a tone that assumes anyone sufficiently rational should already be on board. Well, no! It's not an obvious, open-and shut deal.
What if there were an organization comprised of idealistic, speculative types, who, unknowingly, got themselves to believe something completely false based on sketchy philosophical arguments? They might look a lot like SIAI. Could an outside observer distinguish fruitful non-mainstream speculation from pointless non-mainstream speculation?
I think they are working on their "academic credentials":
http://singinst.org/grants/challenge
...lists some 13 academic papers under various stages of development.
Thanks for that last link. The paper on Changing the frame of AI futurism is extremely relevant to this series of posts.