= 27d567bc2d48d9f40e9ccc20ea370b15)
RobinHanson comments on Less Wrong Q&A with Eliezer Yudkowsky: Ask Your Questions - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (682)
Why exactly do majorities of academic experts in the fields that overlap your FAI topic, who have considered your main arguments, not agree with your main claims?
I also disagree with the premise of Robin's claim. I think that when our claims are worked out precisely and clearly, a majority agree with them, and a supermajority of those who agree with Robin's part (new future growth mode, get frozen...) agree.
Still, among those who take roughly Robin's position, I would say that an ideological attraction to libertarianism is BY FAR the main reason for disagreement. Advocacy of a single control system just sounds too evil for them to bite that bullet however strong the arguments for it.
Which claims? The SIAI collectively seems to think some pretty strange things to me. Many are to do with the scale of the risk facing the world.
Since this is part of its funding pitch, one obvious explanation seems to be that the organisation is attempting to create an atmosphere of fear - in the hope of generating funding.
We see a similar phenomenon surrounding global warming alarmism - those promoting the idea of there being a large risk have a big overlap with those who benefit from related funding.
You would expect serious people who believed in a large risk to seek involvement, which would lead the leadership of any such group to benefit from funding.
Just how many people do you imagine are getting rich off of AGI concerns? Or have any expectation of doing so? Or are even "getting middle class" off of them?
Some DOOM peddlers manage to get by. Probably most of them are currently in Hollywood, the finance world, or ecology. Machine intelligence is only barely on the radar at the moment - but that doesn't mean it will stay that way.
I don't necessarily mean to suggest that these people are all motivated by money. Some of them may really want to SAVE THE WORLD. However, that usually means spreading the word - and convincing others that the DOOM is real and immanent - since the world must first be at risk in order for there to be SALVATION.
Look at Wayne Bent (aka Michael Travesser), for example:
"The End of The World Cult Pt.1"
The END OF THE WORLD - but it seems to have more to do with sex than money.
Any practical advice on how to overcome this failure mode, if and only if it is in fact a failure mode?
Who are we talking about besides you?
I'd consider important overlapping academic fields to be AI and long term economic growth; I base my claim about academic expert opinion on my informal sampling of such folks. I would of course welcome a more formal sampling.
Who's considered my main arguments besides you?
I'm not comfortable publicly naming names based on informal conversations. These folks vary of course in how much of the details of your arguments they understand, and of course you could always set your bar high enough to get any particular number of folks who have understood "enough."
Okay. I don't know any academic besides you who's even tried to consider the arguments. And Nick Bostrom et. al., of course, but AFAIK Bostrom doesn't particularly disagree with me. I cannot refute what I have not encountered, I do set my bar high, and I have no particular reason to believe that any other academics are in the game. I could try to explain why you disagree with me and Bostrom doesn't.
Actually, on further recollection, Steve Omohundro and Peter Cheeseman would probably count as academics who know the arguments. Mostly I've talked to them about FAI stuff, so I'm actually having trouble recalling whether they have any particular disagreement with me about hard takeoff.
I think that w/r/t Cheeseman, I had to talk to Cheeseman for a while before he started to appreciate the potential speed of a FOOM, as opposed to just the FOOM itself which he considered obvious. I think I tried to describe your position to Cheeseman and Cheeseman thought it was pretty implausible, but of course that could just be the fact that I was describing it from outside - that counts for nothing in my view until you talk to Cheeseman, otherwise he's not familiar enough with your arguments. (See, the part about setting the bar high works both ways - I can be just as fast to write off the fact of someone else's disagreement with you, if they're insufficiently familiar with your arguments.)
I'm not sure I can recall what Omohundro thinks - he might be intermediate between yourself and myself...? I'm not sure how much I've talked hard takeoff per se with Omohundro, but he's certainly in the game.
I think Steve Omohundro disagees about the degree to which takeoff is likely to be centralized, due to what I think is the libertarian impulses I mentioned earlier.
Surely some on the recent AAAI Presidential Panel on Long-Term AI Futures considered your arguments to at least some degree. You could discuss why these folks disagree with you.
Haven't particularly looked at that - I think some other SIAI people have. I expect they'd have told me if there was any analysis that counts as serious by our standards, or anything new by our standards.
If someone hasn't read my arguments specifically, then I feel very little need to explain why they might disagree with me. I find myself hardly inclined to suspect that they have reinvented the same arguments. I could talk about that, I suppose - "Why don't other people in your field invent the same arguments you do?"
You have written a lot of words. Just how many of your words would someone have had to read to make you feel a substantial need to explain the fact they are world class AI experts and disagree with your conclusions?
I'm sorry, but I don't really have a proper lesson plan laid out - although the ongoing work of organizing LW into sequences may certainly help with that. It would depend on the specific issue and what I thought needed to be understood about that issue.
If they drew my feedback cycle of an intelligence explosion and then drew a different feedback cycle and explained why it fit the historical evidence equally well, then I would certainly sit up and take notice. It wouldn't matter if they'd done it on their own or by reading my stuff.
E.g. Chalmers at the Singularity Summit is an example of an outsider who wandered in and started doing a modular analysis of the issues, who would certainly have earned the right of serious consideration and serious reply if, counterfactually, he had reached different conclusions about takeoff... with respect to only the parts that he gave a modular analysis of, though, not necessarily e.g. the statement that de novo AI is unlikely because no one will understand intelligence. If Chalmers did a modular analysis of that part, it wasn't clear from the presentation.
Roughly, what I expect to happen by default is no modular analysis at all - just snap consideration and snap judgment. I feel little need to explain such.
I have a theory about why there is disagreement with the AAAI panel:
The DOOM peddlers gather funding from hapless innocents - who hope to SAVE THE WORLD - while the academics see them as bringing their field into disrepute, by unjustifiably linking their field to existential risk, with their irresponsible scaremongering about THE END OF THE WORLD AS WE KNOW IT.
Naturally, the academics sense a threat to their funding - and so write papers to reassure the public that spending money on this stuff is Really Not As Bad As All That.
Me - if I qualify as an academic expert is another matter entirely of course.
Do you disagree with Eliezer substantively? If so, can you summarize how much of his arguments you've analyzed, and where you reach different conclusions?
Yes - I disagree with Eliezer and have analyzed a fair bit of his writings although the style in which it is presented and collected here is not exactly conducive to that effort. Feel free to search for my blog for a detailed analysis and a summary of core similarities and differences in our premises and conclusions.
Assuming I have the correct blog, these two are the only entries that mention Eliezer by name.
Edit: The second entry doesn't mention him, actually. It comes up in the search because his name is in a trackback.
Re: "Assumption A: Human (meta)morals are not universal/rational. Assumption B: Human (meta)morals are universal/rational.
Under assumption A one would have no chance of implementing any moral framework into an AI since it would be undecidable which ones they were." (source: http://rationalmorality.info/?p=112)
I think we've been over that already. For example, Joe Bloggs might choose to program Joe's preferences into an intelligent machine - to help him reach his goals.
I had a look some of the other material. IMO, Stefan acts in an authoritative manner, but comes across as a not-terribly articulate newbie on this topic - and he has adopted what seems to me to be a bizarre and indefensible position.
For example, consider this:
"A rational agent will always continue to co-exist with other agents by respecting all agents utility functions irrespective of their rationality by striking the most rational compromise and thus minimizing opposition from all agents." http://rationalmorality.info/?p=8
"I think we've been over that already. For example, Joe Bloggs might choose to program Joe's preferences into an intelligent machine - to help him reach his goals."
Sure - but it would be moral simply by virtue of circular logic and not objectively. That is my critique.
I realize that one will have to drill deep into my arguments to understand and put them into the proper context. Quoting certain statements out of context is definitely not helpful, Tim. As you can see from my posts, everything is linked back to a source were a particular point is made and certain assumptions are being defended.
If you have a particular problem with any of the core assumptions and conclusions I prefer you voice them not as a blatant rejection of an out of context comment here or there but based on the fundamentals. Reading my blogs in sequence will certainly help although I understand that some may consider that an unreasonable amount of time investment for what seems like superficial nonsense on the surface.
Where is your argument against my points Tim? I would really love to hear one, since I am genuinely interested in refining my arguments. Simply quoting something and saying "Look at this nonsense" is not an argument. So far I only got an ad hominem and an argument from personal incredulity.
From the second blog entry linked above:
Heh.
This quotation accurately summarizes the post as I understand it. (It's a short post.)
I think I speak for many people when I say that assumption A requires some evidence. It may be perfectly obvious, but a lot of perfectly obvious things aren't true, and it is only reasonable to ask for some justification.
I was going to be nice and not say anything, but, yeah.