Sewing-Machine comments on Should I believe what the SIAI claims? - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (600)
Most of the long arguments are concerned with refuting fallacies and defeating counterarguments, which flawed reasoning will always be able to supply in infinite quantity. The key predictions, when you look at them, generally turn out to be antipredictions, and the long arguments just defeat the flawed priors that concentrate probability into anthropomorphic areas. The positive arguments are simple, only defeating complicated counterarguments is complicated.
"Fast AI" is simply "Most possible artificial minds are unlikely to run at human speed, the slow ones that never speed up will drop out of consideration, and the fast ones are what we're worried about."
"UnFriendly AI" is simply "Most possible artificial minds are unFriendly, most intuitive methods you can think of for constructing one run into flaws in your intuitions and fail."
MWI is simply "Schrodinger's equation is the simplest fit to the evidence"; there are people who think that you should do something with this equation other than taking it at face value, like arguing that gravity can't be real and so needs to be interpreted differently, and the long arguments are just there to defeat them.
The only argument I can think of that actually approaches complication is about recursive self-improvement, and even there you can say "we've got a complex web of recursive effects and they're unlikely to turn out exactly exponential with a human-sized exponent", the long arguments being devoted mainly to defeating the likes of Robin Hanson's argument for why it should be exponential with an exponent that smoothly couples to the global economy.
This is quite helpful, and suggests that what I wanted is not a lay-reader summary, but an executive summary.
I brought this up elsewhere in this thread, but the fact that quantum mechanics and gravity are not reconciled suggests that even Schrodinger's equation does not fit the evidence. The "low-energy" disclaimer one has to add is very weird, maybe weirder than any counterintuitive consequences of quantum mechanics.
It's not the Schrödinger equation alone that gives rise to decoherence and thus many-worlds. (Read Good and Real for another toy model, the "quantish" system.) The EPR experiment and Bell's inequality can be made to work on macroscopic scales, so we know that whatever mathematical object the universe will turn out to be, it's not going to go un-quantum on us again: it has the same relevant behavior as the Schrödinger equation, and accordingly MWI will be the best interpretation there as well.