= 27d567bc2d48d9f40e9ccc20ea370b15)
Will_Newsome comments on Intellectual Hipsters and Meta-Contrarianism - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (323)
/startrant
Or, more realistically, the idea that DOOM has a CHANCE of happening any time between NOW and ONE HUNDRED YEARS FROM NOW but that small CHANCE has a large enough impact in EXPECTED UTILITY that we should really figure out more about the problem because someone, not necessarily SIAI might have to deal with the problem EVENTUALLY.
One might naively expect such an organization would typically act so as to exaggerate the risks -- but SIAI doesn't seem to be doing that so one's naive expectations would be wrong. It's amazing how people associate an aura of overconfidence coming from the philosophical positions of Eliezer with the actual confidence levels of the thinkers of SIAI. Seriously, where are these crazy claims about DOOM being SOON and that ELIEZER YUDKOWSKY is the MESSIAH? From something Eliezer wrote 10 years ago? The Singularity Institute is pretty damn reasonable. The journal and conference papers they write are pretty well grounded in sound and careful reasoning. But ha, who would read those? It's not like it'd be a good idea to actually read an organization's actual literary output before judging them based primarily on the perceived arrogance of one of their research fellows, that'd be stupid.
What vested interests? Money? Do you honestly think that the people at SIAI couldn't get 5 times as much money by working elsewhere? Status? Do you honestly think that making a seemingly crazy far mode belief that pattern matches to doomsdayism part of your identity for little pay and lots of hard work is a good way of gaining status? Eliezer would take a large status hit if he admitted he was wrong about this whole seed AI thing. Michael Vassar would too. But everyone else? Really good thinkers like Anna Salamon and Carl Shulman and Steve Rayhawk who have proved here on Less Wrong that they have exceptionally strong rationality, and who are consistently more reasonable than they have any right to be? (Seriously, you could give Steve Rayhawk the most retarded argument ever and he'd find a way to turn it into a reasonable argument worth seriously addressing. These people take their epistemology seriously.)
Maybe people at SIAI are, you know, actually worried about the problems because they know how to take ideas seriously instead of using the absurdity heuristic and personal distaste for Eliezer and then rationalizing their easy beliefs with vague outside view reference class tennis games or stupid things like that.
I like reading Multifoliaterose's posts. He raises interesting points, even if I think they're generally unfair. I can tell that he's at least using his brain. When most people criticize SIAI (really Eliezer, but it's easier to say SIAI 'cuz it feels less personal), they don't use any parts of their brain besides the 'rationalize reason for not associating with low status group' cognitive module.
timtyler, this comment isn't really a direct reply to yours so much as a venting of general frustrations. But I get annoyed by the attitude of 'haha let's be cynical and assume the worst of the people that are actually trying their hardest to do the most good they can for the world'. Carl Shulman would never write a reply anything like the one I've written. Carl Shulman is always reasonable and charitable. And I know Carl Shulman works incredibly hard on being reasonable, and taking into account opposing viewpoints, and not letting his affiliation with SIAI cloud his thinking, and still doing lots of good, reasonable, solid work on explaining the problem of Friendliness to the academic sphere in reasonable, solid journal articles and conference papers.
It's really annoying to me to have that go completely ignored just because someone wants to signal their oh-so-metacontrarian beliefs about SIAI. Use epistemic hygiene. Think before you signal. Don't judge an entire organization's merit off of stupid outside view comparisons without actually reading the material. Take the time to really update on the beliefs of longtime x-rationalists that have probably thought about this a lot more than you have. If you really think it through and still disagree, you should have stronger and more elegant counterarguments than things like "they have glaringly-obvious vested interests". Yeah, as if that didn't apply to anyone, especially anyone who thinks that we're in great danger and should do something about it. They have pretty obvious vested interests in telling people about said danger. Great hypothesis there chap. Great way to rationalize your desire to signal and do what is easy and what appeals to your vanity. Care to list your true rejections?
And if you think that I am being uncharitable in my interpretation of your true motivations, then be sure to notice the symmetry.
/endrant
That was quite a rant!
I hope I don't come across as thinking "the worst" about those involved. I expect they are all very nice and sincere. By way of comparison, not all cults have deliberately exploitative ringleaders.
Really? Really? You actually think the level of DOOM is cold realism - and not a ploy to attract funding? Why do you think that? De Garis and Warwick were doing much the same kind of attention-seeking before the SIAI came along - DOOM is an old school of marketing in the field.
You encourage me to speculate about the motives of the individuals involved. While that might be fun, it doesn't seem to matter much - the SIAI itself is evidently behaving as though it wants dollars, attention, and manpower - to help it meet its aims.
FWIW, I don't see what I am saying as particularly "contrarian". A lot of people would be pretty sceptical about the end of the world being nigh - or the idea that a bug might take over the world - or the idea that a bunch of saintly programmers will be the ones to save us all. Maybe contrary to the ideas of the true believers - if that is what you mean.
Anyway, the basic point is that if you are interested in DOOM, or p(DOOM), consulting a DOOM-mongering organisation, that wants your dollars to help them SAVE THE WORLD may not be your best move. The "follow the money" principle is simple - and often produces good results.
Right, I said metacontrarian. Although most LW people seem SIAI-agnostic, a lot of the most vocal and most experienced posters are pro-SIAI or SIAI-related, so LW comes across as having a generally pro-SIAI attitude, which is a traditionally contrarian attitude. Thus going against the contrarian status quo is metacontrarian.
I'm confused. Anyone trying to accomplish anything is going to try to get dollars, attention, and manpower. I'm confused as to how this is relevant to the merit of SIAI's purpose. SIAI's never claimed to be fundamentally opposed to having resources. Can you expand on this?
What makes that comparison spring to mind? Everyone is incredibly critical of Eliezer, probably much more so than he deserves, because everyone is racing to be first to establish their non-cult-victim status. Everyone at SIAI has different beliefs about the relative merits of different strategies for successful FAI development. That isn't a good thing -- fractured strategy is never good -- but it is evidence against cultishness. SIAI grounds its predictions in clear and careful epistemology. SIAI publishes in academic journals, attends scientific conferences, and hosts the Singularity Summit, where tons of prominent high status folk show up to speak about Singularity-related issues. Why is cult your choice of reference class? It is no more a cult than a typical global warming awareness organization. It's just that 'science fiction' is a low status literary genre in modern liberal society.
I don't know about anybody else, but I am somewhat disturbed by Eliezer's persistent use of hyphens in place of em dashes, and am very concerned that it could be hurting SIAI's image.
And I say the same about his use of double spacing. It's an outdated and unprofessional practice. In fact, Anna Salamon and Louie Helm are 2 other SIAI folk that engage in this abysmal writing style, and for that reason I've often been tempted to write them off entirely. They're obviously not cognizant of the writing style of modern academic thinkers. The implications are obvious.
Another reason that I suspect is more important than trying to signal non-cult-victim status is that people who do want to be considered part of the cult believe that the cause is important and believe that Eliezer's mistakes could destroy the world (for example).
To recap, the SIAI is funded by donations from those who think that they will help prevent the end of the world at the hands of intelligent machines. For this pitch to work, the world must be at risk - in order for them to be able to save it. The SIAI face some resistance over this point, and these days, much of their output is oriented towards convincing others that these may be the end days. Also there will be a selection bias, with those most convinced of a high p(DOOM) most likely to be involved. Like I said, not necessarily the type of organisation one would want to approach if seeking the facts of the matter.
You pretend to fail to see connections between the SIAI and an END OF THE WORLD cult - but it isn't a terribly convincing act.
For the connections, see here. For protesting too much, see You're calling who a cult leader?
No, I see it, look further, and find the model lacking in explanatory power. It selectively leaves out all kinds of useful information that I can use to control my anticipations.
Hmuh, I guess we won't be able to make progress, 'cuz I pretty much wholeheartedly agree with Vladimir when he says:
and Nick Tarleton when he says:
"This one is right" for example. ;)
I didn't say anyone was "racing to be first to establish their non-cult-victim status" - but it is certainly a curious image! [deleted parent comment was a dupe].
Oops, connection troubles then missed.
Tim, do you think that nuclear-disarmament organizations were inherently flawed from the start because their aim was to prevent a catastrophic global nuclear war? Would you hold their claims to a much higher standard than the claims of organizations that looked to help smaller numbers of people here and now?
I recognize that there are relevant differences, but merely pattern-matching an organization's conclusion about the scope of their problem, without addressing the quality of their intermediate reasoning, isn't sufficient reason to discount their rationality.
Will said "meta-contrarian," which refers to the recent meta-contrarians are intellectual hipsters post.
I also think you see yourself as trying to help SIAI see how they look to "average joe" potential collaborators or contributors, while Will sees your criticisms as actually calling into question the motives, competence, and ingenuity of SIAI's staff. If I'm right, you're talking at cross-purposes.
Reforming the SIAI is a possibility - but not a terribly realistic one, IMO. So, my intended audience here is less that organisation, and more some of the individuals here who I share interests with.
Oh, that might be. Other comments by timtyler seemed really vague but generally anti-SIAI (I hate to set it up as if you could be for or against a set of related propositions in memespace, but it's natural to do here, meh), so I assumed he was expressing his own beliefs, and not a hypothetical average joe's.