Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Eliezer_Yudkowsky comments on The Importance of Self-Doubt - Less Wrong

23 Post author: multifoliaterose 19 August 2010 10:47PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (726)

You are viewing a single comment's thread.

Comment author: Eliezer_Yudkowsky 20 August 2010 03:46:39AM 20 points [-]

Unknown reminds me that Multifoliaterose said this:

The modern world is sufficiently complicated so that no human no matter how talented can have good reason to believe himself or herself to be the most important person in human history without actually doing something which very visibly and decisively alters the fate of humanity. At present, anybody who holds such a belief is suffering from extreme delusions of grandeur.

This makes explicit something I thought I was going to have to tease out of multi, so my response would roughly go as follows:

  • If no one can occupy this epistemic state, that implies something about the state of the world - i.e., that it should not lead people into this sort of epistemic state.
  • Therefore you are deducing information about the state of the world by arguing about which sorts of thoughts remind you of your youthful delusions of messianity.
  • Reversed stupidity is not intelligence. In general, if you want to know something about how to develop Friendly AI, you have to reason about Friendly AI, rather than reasoning about something else.
  • Which is why I have a policy of keeping my thoughts on Friendly AI to the object level, and not worrying about how important or unimportant that makes me. In other words, I am reluctant to argue on this level not just for the obvious political reasons (it's a sure loss once the argument starts), but because you're trying to extract information about the real world from a class of arguments that can't possibly yield information about the real world.
  • That said, as far as I can tell, the world currently occupies a ridiculous state of practically nobody working on problems like "develop a reflective decision theory that lets you talk about self-modification". I agree that this is ridiculous, but seriously, blame the world, not me. Multi's principle would be reasonable only if the world occupied a much higher level of competence than it in fact does, a point which you can further appreciate by, e.g., reading the QM sequence, or counting cryonics signups, showing massive failure on simpler issues.
  • That reflective decision theory actually is key to Friendly AI is something I can only get information about by thinking about Friendly AI. If I try to get information about it any other way, I'm producing noise in my brain.
  • We can directly apply multi's stated principle to conclude that reflective decision theory cannot be known to be critical to Friendly AI. We were mistaken to start working on it; if no one else is working on it, it must not be knowably critical; because if it were knowably critical, we would occupy a forbidden epistemic state.
  • Therefore we have derived knowledge about which problems are critical in Friendly AI by arguing about personal psychology.
  • This constitutes a reductio of the original principle. QEA. (As was to be argued.)
Comment author: Jonathan_Graehl 20 August 2010 04:18:46AM *  4 points [-]

Upvoted for being clever.

You've (probably) refuted the original statement as an absolute.

You're deciding not to engage the issue of hubris directly.

Does the following paraphrase your position:

  1. Here's what I (and also part of SIAI) intend to work on

  2. I think it's very important (and you should think so for reasons outline in my writings)

  3. If you agree with me, you should support us

? If so, I think it's fine for you to not say the obvious (that you're being quite ambitious, and that success is not assured). It seems like some people are really dying to hear you say the obvious.

Comment author: Eliezer_Yudkowsky 20 August 2010 05:03:09AM 9 points [-]

Success is not assured. I'm not sure what's meant by confessing to being "ambitious". Is it like being "optimistic"? I suppose there are people who can say "I'm being optimistic" without being aware that they are instantiating Moore's Paradox but I am not one of them.

I also disclaim that I do not believe myself to be the protagonist, because the world is not a story, and does not have a plot.

Comment author: Perplexed 20 August 2010 05:14:49AM 1 point [-]

I hope that the double negative in the last sentence was an error.

I introduced the term "protagonist", because at that point we were discussing a hypothetical person who was being judged regarding his belief in a set of three propositions. Everyone recognized, of course, who that hypothetical person represented, but the actual person had not yet stipulated his belief in that set of propositions.

Comment author: wedrifid 20 August 2010 10:57:37AM 2 points [-]

I hope that the double negative in the last sentence was an error.

Interesting. I don't claim great grammatical expertise but my reading puts the last question at reasonable. Am I correct in inferring that you do not believe Eliezer's usage of "I also disclaim" to mean "I include the following disclaimer: " is valid?

Regarding 'protagonist' there is some context for the kind of point Eliezer likes to make about protagonist/story thinking in his Harry Potter fanfic. I don't believe he has expressed the concept coherently as a post yet. (I don't see where you introduced the 'protagonist' word so don't know whether Eliezer read you right. I'm just throwing some background in.)

Comment author: Perplexed 20 August 2010 07:01:47PM 3 points [-]

Regarding "disclaim".

I read "disclaim" as a synonym for "deny". I didn't even consider your interpretation, but upon consideration, I think I prefer it.

My mistake (again!). :(

Comment author: Vaniver 28 November 2010 08:02:09PM *  1 point [-]

Am I correct in inferring that you do not believe Eliezer's usage of "I also disclaim" to mean "I include the following disclaimer: " is valid?

This question is best solved by a dictionary. "I disclaim that I am a blegg" means that I am not a blegg; "Disclaimer: I am a blegg" means that I am a blegg. The use of disclaimer in the second statement is describing the following statement: "I am making a claim that denies something: I am a blegg."

Take home message: Eliezer's double negative means his post has the opposite effect of what I hope he intended.

Comment author: Jonathan_Graehl 20 August 2010 10:02:35PM 0 points [-]

Yes, that was exactly the sense of "ambitious" I intended - the second person sneering one, which when used by oneself, would be more about signaling humility than truth. I see that's not your style.

Comment author: wedrifid 20 August 2010 10:08:09AM *  14 points [-]

Upvoted for being clever.

That's interesting. I downvoted it for being clever. It was a convoluted elaboration of a trivial technicality that only applies if you make the most convenient (for Eliezer) interpretation of multi's words. This kind of response may win someone a debating contest in high school but it certainly isn't what I would expect from someone well versed in the rationalism sequences, much less their author.

I don't pay all that much attention to what multi says (no offence intended to multi) but I pay close attention to what Eliezer does. I am overwhelmingly convinced of Eliezer's cleverness and brilliance as a rationalism theorist. Everything else, well, that's a lot more blurry.

Comment author: Furcas 20 August 2010 10:31:53AM *  2 points [-]

I don't think Eliezer was trying to be clever. He replied to the only real justification multi offered for why we should believe that Eliezer is suffering from delusions of grandeur. What else is he supposed to do?

Comment author: wedrifid 20 August 2010 12:00:48PM 5 points [-]

I got your reply and respect your position. I don't want to engage too much here since it would overlap with discussion surrounding Eliezer's initial reply and potentially be quite frustrating.

What I would like to see is multifoliaterose giving a considered response to the "If not, why not?" question in that link. That would give Eliezer the chance to respond to the meat of the topic at hand. Eliezer has been given a rare opportunity. He can always write posts about himself, giving justifications for whatever degree of personal awesomeness he claims. That's nothing new. But in this situation it wouldn't be perceived as Eliezer grabbing the megaphone for his own self-gratification. He is responding to a challenge, answering a request.

Why would you waste the chance to, say, explain the difference between "SIAI" and "Eliezer Yudkowsky"? Or at least give some treatment of p(someone other than Eliezer Yudkowsky is doing the most to save the world). Better yet, take that chance to emphasise the difference between p(FAI is the most important priority for humanity) and p(Eliezer is the most important human in the world).

Comment author: khafra 20 August 2010 04:49:22PM 1 point [-]

As Graehl and wedrifid observed, Eliezer responded as if the original statement were an absolute. He applied deductive reasoning and found a reductio ad absurdum. But if, instead of an absolute, you see multifoliaterose's characterization as a reference class: "People who believe themselves to be one of the few most important in the world without having already done something visible and obvious to dramatically change it," it can lower the probability that Eliezer is, in fact, that important by a large likelihood ratio.

Whether this likelihood ratio is large enough to overcome the evidence on AI-related existential risk and the paucity of serious effort dedicated to combating it is an open question.

Comment author: Unknowns 20 August 2010 09:27:13AM *  1 point [-]

Even if almost everything you say here is right, it wouldn't mean that there is a high probability that if you are killed in a car accident tomorrow, no one else will think about these things (reflective decision theory and so on) in the future, even people who know nothing about you personally. As Carl Shulman points out, if it is necessary to think about these things it is likely that people will, when it becomes more urgent. So it still wouldn't mean that you are the most important person in human history.

Comment author: multifoliaterose 20 August 2010 06:39:52PM 0 points [-]

I agree with khafra. Your response to my post is distortionary. The statement which you quote was a statement about the reference class of people who believe themselves to be the most important person in the world. The statement which you quote was not a statement about FAI.

Any adequate response to the statement which you quote requires that you engage with the last point that khafra made:

Whether this likelihood ratio is large enough to overcome the evidence on AI-related existential risk and the paucity of serious effort dedicated to combating it is an open question.

You have not satisfactorily addressed this matter.

Comment author: Furcas 21 August 2010 03:36:59PM *  4 points [-]

It looks to me like Eliezer gave your post the most generous interpretation possible, i.e. that it actually contained an argument attempting to show that he's deluding himself, rather than just defining a reference class and pointing out that Eliezer fits into it. Since you've now clarified that your post did nothing more than that, there's not much left to do except suggest you read all of Eliezer's posts tagged 'FAI', and this.