= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on A cynical explanation for why rationalists worry about FAI - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (179)
Let's take the outside view for a second. After all, if you want to save the planet from AIs, you have to do a lot of thinking! You have to learn all sorts of stuff and prove it and just generally solve a lot of eye-crossing philosophy problems which just read like slippery bullshit. But if you want to save the planet from asteroids, you can conveniently do the whole thing without ever leaving your own field and applying all the existing engineering and astronomy techniques. Why, you even found a justification for NASA continuing to exist (and larding out pork all over the country) and better yet, for the nuclear weapons program to be funded even more (after all, what do you think you'll be doing when the Shuttle gets there?).
Obviously, this isn't any sort of proof that anti-asteroid programs are worthless self-interested rent-seeking government pork.
But it sure does seem suspicious that continuing business as usual to the tune of billions can save the entire species from certain doom.
To be fair though, a lot of us would learn the tricky philosophy stuff anyway just because it seems interesting. It is pretty possible that our obsession with FAI stems partially from the fact that the steps needed to solve such a problem appeal to us. Not to say that FAI isn't EXTREMELY important by its own merits, but there are a number of existential risks that pose relatively similar threat levels that we don't talk about night and day.
My actual take is that UFAI is actually a much larger threat than other existential risks, but also that working on FAI is fairly obviously the chosen path, not on EV grounds, but on the grounds of matching our skills and interests.
Yes, I agree that if a politician or government official tells you the most effective thing you can do to prevent asteroids from destroying the planet is "keep NASA at current funding levels and increase funding for nuclear weapons research" then you should be very suspicious.
I think you're missing the point; I actually do think NASA is one of the best organizations to handle anti-asteroid missions and nukes are a vital tool since the more gradual techniques may well take more time than we have.
Your application of cynicism proves everything, and so proves nothing. Every strategy can be - rightly - pointed out to benefit some group and disadvantage some other group.
The only time this wouldn't apply is if someone claiming a particular risk is higher than estimated and was doing absolutely nothing about it whatsoever and so couldn't benefit from attempts to address it. And in that case, one would be vastly more justified in discounting them because they themselves don't seem to actually believe it rather than believing them because this particular use of Outside View doesn't penalize them.
(Or to put it another more philosophical way: what sort of agent believes that X is a valuable problem to work on, and also doesn't believe that whatever Y approach he is taking is the best approach for him to be taking? One can of course believe that there are better approaches for other people - 'if I were a mathematical genius, I could be making more progress on FAI than if I were an ordinary person whose main skills are OK writing and research' - or for counterfactual selves with stronger willpower, but for oneself? This is analogous to Moore's paradox or the epistemic question, what sort of agent doesn't believe that his current beliefs are the best for him to hold? "It's raining outside, but I don't believe it is." So this leads to a remarkable result: for every agent which is trying to accomplish something, we can cynically say 'how very convenient that the approach you think is best is the one you happen to be using! How awfully awfully convenient! Not.' And since we can say it for every agent equally, the argument is entirely useless.)
Incidentally:
I think you badly overstate your case here. Most armchair rationalists seem to much prefer activities like... saving the world by debunking theism (again). How many issues have Skeptic or Skeptical Inquirer devoted to discussing FAI?
There's a much more obvious reason why many LWers would find FAI interesting other than the concept being some sort of attractive death spiral for armchair rationalists in general...
FHI, for what it's worth, does say that simulation shutdown is underestimated but doesn't suggest doing anything.
My suspicion isn't because the recommended strategy has some benefits, it's because it has no costs. It would not be surprising if an asteroid-prevention plan used NASA and nukes. It would be surprising if it didn't require us to do anything particularly hard. What's suspicious about SIAI is how often their strategic goals happen to be exactly the things you might suspect the people involved would enjoy doing anyway (e.g. writing blog posts promoting their ideas) instead of difficult things at which they might conspicuously fail.
"But it sure does seem suspicious that continuing business as usual can save the entire species from certain doom."
Doesn't this sentence apply here? What exactly is this community doing that's so unusual (other than giving EY money)?
The frame of "saving humanity from certain doom" seems to serve little point other than a cynical way of getting certain varieties of young people excited.
As far as I can tell, SI long ago started avoiding that frame because the frame had deleterious effects, but if we wanted to excite anyone, it was ourselves, not other young people.
Exploring many unusual and controversial ideas? Certainly we get criticized for focusing on things like FAI often enough, it should at least be true!
Saying that you save the world by exploring many unusual and controversial ideas is like saying you save the world by eating ice cream and playing video games.
Isn't "exploring many unusual and controversial ideas" what scientists usually do? (Ok, maybe sometimes good scientist do it...) Don't you think that science could contribute to saving the world?
What I am saying is "exploring unusual and controversial ideas" is the fun part of science (along with a whole lot of drudgery). You don't get points for doing fun things you would rather be doing anyways.
Actually, I think you get points for doing things that work, whether they are fun or not.
Some of the potentially useful soft sciences research is controversial. But essentially no hard sciences research is both (a) controversial and (b) likely to contribute massive improvement in human well-being.
Even something like researching the next generation of nuclear power plants is controversial only in the sense that all funding of basic research is "controversial."
Nuclear science is controversial for the same reason that equal-access marriage is controversial: Because there are people who have some opinions that cannot be changed by rational argument.
<moar cynicism>If the real reason people want nuclear power plants were their benefits compared to other ways of generating power, they'd use thorium not uranium.</moar cynicism> http://en.wikipedia.org/wiki/User:RaptorHunter/FunFacts#Thorium_reactor
[citation needed]
No, a user's talk page won't do.
Over-hyped BS. There are regular reactors, and there are breeder reactors (fast neutron reactors), both can use uranium but only the latter type can use thorium. The latter type, also, incidentally, uses a lot less uranium than the former, and can use depleted uranium. The cost of fuel is of no consequence and all the safety issues are virtually identical for fast neutron reactors using thorium and using uranium (and for both, are expected to be significantly more severe than for regular water moderated reactors) There's a lot of depleted uranium laying around costing negative $ . Not thorium, though.
I thought that depleted uranium wrapped in paper was pretty much as safe as lead?
There's some ambiguity in your use of the word "science." Nuclear engineering is controversial (i.e. building and running nuclear plants is politically controversial).
But the post I was responding to was about research. In terms of political controversy, I suspect that nuclear researchers receive essentially no hate mail, especially compared to sociologists researching child-rearing outcomes among opposite-sex and same-sex couples.
Nuclear researchers run reactors. It's pretty much the only commercially viable way to test the effects of neutron bombardment on materials. They typically aren't power plants, because steam turbines are a lot more work to operate and research reactors are typically intermittent and low power (in terms of the electrical grid)