In the past, people like Eliezer Yudkowsky (see 1, 2, 3, 4, and 5) have argued that MIRI has a medium probability of success. What is this probability estimate based on and how is success defined?
I've read standard MIRI literature (like "Evidence and Import" and "Five Theses"), but I may have missed something.
-
(Meta: I don't think this deserves a discussion thread, but I posted this on the open thread and no-one responded, and I think it's important enough to merit a response.)
Not easily. Antiantiheroic epistemology might be a better term, i.e., I think that a merely accurate epistemology doesn't have a built-in mechanism which prevents people from thinking they can do things because the outside view says it's nonvirtuous to try to distinguish yourself within reference class blah. Antiantiheroic epistemology doesn't say that it's possible to distinguish yourself within reference class blah so much as it thinks that the whole issue is asking the wrong question and you should mostly be worrying about staying engaged with the object-level problem because this is how you learn more and gain the ability to take opportunities as they arrive. An antiheroic epistemology that throws up some reference class or other saying this is impossible will regard you as trying to distinguish yourself within this reference class, but this is not what the antiantiheroic epistemology is actually about; that's an external indictment of nonvirtuosity arrived at by additional modus ponens to conclusions on which antiantiheroic epistemology sees no reason to expend cognitive effort.
Obviously from my perspective non-antiheroic epistemology cancels out to mere epistemology, simpler for the lack of all this outside-view-social-modesty wasted motion, but to just go around telling you "That's not how epistemology works, of course!" would be presuming a known standard which is logically rude (I think you are doing this, though not too flagrantly).
An archetypal example of antiantiheroic epistemology is Harry in Methods of Rationality, who never bothers to think about any of this reference class stuff or whether he's being immodest, just his object-level problems in taking over the universe, except once when Hermione challenges him on it and Harry manages to do one thing a normal wizard can't. Harry doesn't try to convince himself of anything along those lines, or think about it without Hermione's prompting. It just isn't something that occurs to him might be a useful thought process.
I don't think it's a useful thought process either, and rationalizing elaborate reasons why I'm allowed to be a hero wouldn't be useful either (Occam's Imaginary Razor: decorating my thought processes with supportive tinsel will just slow down any changes I need to make), which is why I tend to be annoyed by the entire subject and wish people would get back to the object level instead of meta demands for modesty that come with no useful policy suggestions about ways to do anything better. Tell me a better object-level way to save the world and we can talk about my doing that instead.
"Antiantiheroic epistemology might be a better term, i.e., I think that a merely accurate epistemology doesn't have a built-in mechanism which prevents people from thinking they can do things because the outside view says it's nonvirtuous to try to distinguish yourself within reference class blah. "
Taken literally, I can't possibly disagree with this, but it doesn't seem to answer my question, which is "where is the positive evidence that one is not supposed to ignore." I favor combining many different kinds of evidence, including spars... (read more)