The Singularity Institute's Arrogance Problem

lukeprog

84 The Singularity Institute's Arrogance Problem

by lukeprog

18th Jan 2012

1 min read

307

84

I intended Leveling Up in Rationality to communicate this:

Despite worries that extreme rationality isn't that great, I think there's reason to hope that it can be great if some other causal factors are flipped the right way (e.g. mastery over akrasia). Here are some detailed examples I can share because they're from my own life...

But some people seem to have read it and heard this instead:

I'm super-awesome. Don't you wish you were more like me? Yay rationality!

This failure (on my part) fits into a larger pattern of the Singularity Institute seeming too arrogant and (perhaps) being too arrogant. As one friend recently told me:

At least among Caltech undergrads and academic mathematicians, it's taboo to toot your own horn. In these worlds, one's achievements speak for themselves, so whether one is a Fields Medalist or a failure, one gains status purely passively, and must appear not to care about being smart or accomplished. I think because you and Eliezer don't have formal technical training, you don't instinctively grasp this taboo. Thus Eliezer's claim of world-class mathematical ability, in combination with his lack of technical publications, make it hard for a mathematician to take him seriously, because his social stance doesn't pattern-match to anything good. Eliezer's arrogance as evidence of technical cluelessness, was one of the reasons I didn't donate until I met [someone at SI in person]. So for instance, your boast that at SI discussions "everyone at the table knows and applies an insane amount of all the major sciences" would make any Caltech undergrad roll their eyes; your standard of an "insane amount" seems to be relative to the general population, not relative to actual scientists. And posting a list of powers you've acquired doesn't make anyone any more impressed than they already were, and isn't a high-status move.

So, I have a few questions:

What are the most egregious examples of SI's arrogance?
On which subjects and in which ways is SI too arrogant? Are there subjects and ways in which SI isn't arrogant enough?
What should SI do about this?

Machine Intelligence Research Institute (MIRI)SignalingSocial Status

Frontpage

84

New Comment

Rendering 0/307 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 2:14 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

Moderation Log

84 The Singularity Institute's Arrogance Problem

by lukeprog

18th Jan 2012

1 min read

307

84

I intended Leveling Up in Rationality to communicate this:

Despite worries that extreme rationality isn't that great, I think there's reason to hope that it can be great if some other causal factors are flipped the right way (e.g. mastery over akrasia). Here are some detailed examples I can share because they're from my own life...

But some people seem to have read it and heard this instead:

I'm super-awesome. Don't you wish you were more like me? Yay rationality!

This failure (on my part) fits into a larger pattern of the Singularity Institute seeming too arrogant and (perhaps) being too arrogant. As one friend recently told me:

At least among Caltech undergrads and academic mathematicians, it's taboo to toot your own horn. In these worlds, one's achievements speak for themselves, so whether one is a Fields Medalist or a failure, one gains status purely passively, and must appear not to care about being smart or accomplished. I think because you and Eliezer don't have formal technical training, you don't instinctively grasp this taboo. Thus Eliezer's claim of world-class mathematical ability, in combination with his lack of technical publications, make it hard for a mathematician to take him seriously, because his social stance doesn't pattern-match to anything good. Eliezer's arrogance as evidence of technical cluelessness, was one of the reasons I didn't donate until I met [someone at SI in person]. So for instance, your boast that at SI discussions "everyone at the table knows and applies an insane amount of all the major sciences" would make any Caltech undergrad roll their eyes; your standard of an "insane amount" seems to be relative to the general population, not relative to actual scientists. And posting a list of powers you've acquired doesn't make anyone any more impressed than they already were, and isn't a high-status move.

So, I have a few questions:

What are the most egregious examples of SI's arrogance?
On which subjects and in which ways is SI too arrogant? Are there subjects and ways in which SI isn't arrogant enough?
What should SI do about this?

Machine Intelligence Research Institute (MIRI)SignalingSocial Status

Frontpage

84

New Comment

Rendering 0/307 comments, sorted by

top scoring

(show more) Click to highlight new comments since: Today at 2:14 AM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

Moderation Log

More from lukeprog

Curated and popular this week

307Comments

307

Comment Permalink

Eliezer Yudkowsky14y130

I believe that. My first-pass filter for theories of why some people think SIAI is "arrogant" is whether the theory also explains, in equal quantity, why those same people find Harry James Potter-Evans-Verres to be an unbearably snotty little kid or whatever. If the theory is specialized to SIAI and doesn't explain the large quantities of similar-sounding vitriol gotten by a character in a fanfiction in a widely different situation who happens to be written by the same author, then in all honesty I write it off pretty quickly. I wouldn't mind understanding this better, but I'm looking for the detailed mechanics of the instinctive sub-second ick reaction experienced by a certain fraction of the population, not the verbal reasons they reach for afterward when they have to come up with a serious-sounding justification. I don't believe it, frankly, any more than I believe that someone actually hates hates hates Methods because "Professor McGonagall is acting out of character".

Showing 3 of 4 replies (Click to show all)

CharlesR14y130

I once read a book on characterization. I forget the exact quote, but it went something like, "If you want to make your villian more believable, make him more intelligent."

I thought my brain had misfired. But apparently, for the average reader it works.

2Matt_Simpson14y

FWIW I don't ever recall having this reaction to Harry, though my memory is pretty bad and I think I'm easily manipulated by stories. It may have something to do with being terse and blunt - this often makes the speaker seem as though they think they're "better" than their interlocutors. I had a Polish professor for one of my calculus classes in undergrad who, being a Pole speaking english, naturally sounded very blunt to our American ears. There were several students in that class who just though he was an arrogant asshole who talked down to his students. I'm mostly speculating here though.

2[anonymous]14y

Self-reference and any more than a moderate degree of certainty about anything that isn't considered normal by whoever happens to be listening are both (at least, in my experience) considered less than discreet. Trying to demonstrate that one isn't arrogant probably qualifies as arrogance, too. I don't know how useful this observation is, but I thought it was at least worth posting.

See in context