LESSWRONG
LW

Comment Permalink

I do not approve of the renaming, singularity to intelligence explosion, in this particular context.

Facing the Singu – Intelligence Explosion, is an emotional piece of writing, there are sections about your (Luke’s) own intellectual and emotional journey to singularitarianism, a section about how to overcome whatever quarrels one might have with the truth and the way towards it (Don’t Flinch Away), and finally the utopian ending which obviously is written to have emotional appeal.

The expression “intelligence explosion” does not have emotional appeal. The word intelligence sounds serious, and thus it fits well in, say, the name of a research institute, but many people view intelligence as more or less the opposite of emotion, or at least at something geeky and boring. And while they for sure are wrong in doing so, as also explained in the text, the association still remains. The word “explosion” also has mostly negative connotations.

“Singularity”, on the other hand, has been hyped for decades, by science fiction, by Kurzweil, and by even SIAI before the rebranding. Sci-fi and Kurzweil may not have given the word the most thorough underpinning, but they gave it hype and recognition, and texts such as this could give the needed foundation in reality.

I understand that the renaming is part of the “political” move of distancing the MIRI from some of the hype, but for this particular text, I reckon it a bad choice. “Facing The Singularity” would sell more copies.

See in context

22 Facing the Intelligence Explosion discussion page

by lukeprog

26th Nov 2011

1 min read

138

22

I've created a new website for my ebook Facing the Intelligence Explosion:

Sometime this century, machines will surpass human levels of intelligence and ability, and the human era will be over. This will be the most important event in Earth’s history, and navigating it wisely may be the most important thing we can ever do.

Luminaries from Alan Turing and Jack Good to Bill Joy and Stephen Hawking have warned us about this. Why do I think they’re right, and what can we do about it?

Facing the Intelligence Explosion is my attempt to answer those questions.

This page is the dedicated discussion page for Facing the Intelligence Explosion.

If you'd like to comment on a particular chapter, please give the chapter name at top of your comment so that others can more easily understand your comment. For example:

Re: From Skepticism to Technical Rationality

Here, Luke neglects to mention that...

Intelligence explosion

Personal Blog

22

Facing the Intelligence Explosion discussion page

0David_Kristoffersson

New Comment

138 comments, sorted by

top scoring

Click to highlight new comments since: Today at 8:29 PM

Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

[-]Grognor13y200

Re: this image

Fucking brilliant.

[-]lukeprog13y150

I was relatively happy with that one. It's awfully hard to represent the Singularity visually.

Uncropped, in color, for your pleasure.

3Vladimir_Nesov13y

Where is it from?

7Dufaer13y

From the "About" page:

0katydee13y

Yeah, that's a great image. In my opinion, it would be slightly improved if some of the original fog was drifting in front of/around the towers of the city, as this would both be a nice callback to the original art and help show that the future is still somewhat unclear, but the effort involved might be incommensurate with the results.

2lukeprog13y

I would love to have a digital artist draw something very much like my mashup (but incorporating your suggestion) as an original composition, so I could use it.

2Paul Crowley13y

I note that since this was written you've had a new, closely related image redrawn by Eran Cantrell. I'd actually thought it was Aaron Diaz from the way the guy looks...

0kurzninja12y

Looking at the new image now that I've just finished reading the E-book, I could have sworn it was Benedict Cumberbatch playing Sherlock Holmes.

1Bongo13y

It's also another far-mode picture.

0loup-vaillant13y

Yeah, except the minor quibble I have over what this image actually represents if you have some background knowledge (the space city is from a computer game). But sure, the look of it is absolutely brilliant.

[-][anonymous]13y130

I find this chatty, informal style a lot easier to read than your formal style. The sentences are shorter and easier to follow, and it flows a lot more naturally. For example, compare your introduction to rationality in From Skepticism to Technical Rationality to that in The Cognitive Science of Rationality. Though the latter defines its terms more precisely, the former is much easier to read.

Probably relevant: this comment by Yvain.

[-]James_Miller13y110

Re: Preface

Luke discusses his conversion from Christianity to atheism in the preface. This journey plays a big role in how he came to be interested in the Singularity, but this conversion story might mistakenly lead readers to think that the arguments in favor of believing in a Singularity can also be used to argue against the existence of a God. If you want to get people to think rationally about the future of machine intelligence you might not want to intertwine your discussion with religion.

6Vladimir_Nesov13y

Some of them surely can.

6Shmi13y

name three

[-]Kevin13y160

Kolmogorov Complexity/Occam's Razor
Reductionism/Physicialism
Rampant human cognitive biases

0[anonymous]13y

(1) If the Singularity leads to the destruction of homo sapiens sapiens as a species and a physically and functionally different set of organisms (virtual or material) appears that can comprehend and imagine everything humans could and much much more (or perhaps can't imagine some!), and this claim can be made fairly definitively, I claim it updates "God made mankind in his image" to negligible probability. (2) If theoretical physicists come to a concensus in the ontological context of Penrose’s math-matter-mind triangle (see http://arxiv.org/pdf/physics/0510188v2), such as for example finding a way to agree on the "?" in the mystic approach, and such a consensus requires post-Singularity minds, the question of existence of God dissolves. (3) In the hypothetical case that (i) the picture of humanity as a species arising out of natural selection due to competition between genes is accurate, (ii) we rebel against our genes (see http://hanson.gmu.edu/matrix.html ) and the physical structure of DNA no longer is present on the planet or is somehow fundamentally altered, (iii) the mental concept of "God" is a corollary of particular gene expression, then nobody will be talking or thinking about "the existence of a God", and in some sense "God" will cease to exist as a serious idea and become a cute story, in analogy to alchemy, or demons inducing madness. You can still talk about turning iron into gold, or someone in an epileptic bout being controlled by a demon, but the mainstream of conscious organisms no longer have belief in belief or belief in that sort of talk. A modern medical understanding of certain psychological disorders is functionally equivalent to an argument that demons don't exist. All three examples share a key feature: in some sense, a Singularity might brainwash people (or the information processes descended from people) into believing God does not exist in the same sense that science brainwashed people into believing that alchemy is hogwash. Now re

6Normal_Anomaly13y

I think the target audience mostly consists of atheists, to the point where associating Singularitarianism with Atheism will help more than hurt. Especially because "it's like a religion" is the most common criticism of the Singularity idea. On another note, that paragraph has a typo:

[-]torekp13y170

Especially because "it's like a religion" is the most common criticism of the Singularity idea.

Which is exactly why I worry about a piece that might be read as "I used to have traditional religion. Now I have Singularitarianism!"

0Normal_Anomaly13y

Yes, that is a problem. I was responding to: which I saw as meaning that Singularitarianism might be perceived as associated with atheism. Associating it with atheism would IMO be a good thing, because it's already associated with religion like you said. The question is, does the page as currently written cause people to connect Singularitarianism with religion or atheism?

4Logos0113y

Not if it causes people to associate Singularitarianism as a "religion-substitute for atheists".

4lukeprog13y

Fixed, thanks.

2Kevin13y

I think it's possible that the paragraph could be tweaked in such a way as to make religious people empathize with Luke's religious upbringing rather than be alienated by it.

3lukeprog13y

Any suggestions for how?

[-]Kutta13y50

Re: Preface and Contents

I am somewhat irked by "the human era will be over" phrase. It is not given that current-type humans cease to exist after any Singularity. Also, a positive Singularity could be characterised as the beginning of the humane era, in which case it is somewhat inappropriate to refer to the era afterwards as non-human. In contrast to that, negative Singularities typically result in universes devoid of human-related things.

3Logos0113y

Iron still exists, and is actively used, but we no longer live in the Iron Age.

0Kutta13y

My central point is contained in the sentence after that. A positive Singularity seems extremely human to me when contrasted to paperclip Singularities.

-1Logos0113y

I am not particularly in the habit of describing human beings as "humane". That is a trait of those things which we aspire to but only too-rarely achieve.

[-]stat113y40

I just wanted to say the French translation is of excellent quality. Whoever is writing it, thanks for that. It helps me learn the vocabulary so I can have better discussions with French-speaking people.

[-]lukeprog13y40

Re: Playing Taboo with "Intelligence"

Jaan Tallinn said to me:

i use "given initial resources" instead of "resources used" -- since resource acquisition is an instrumental goal, the future availability of resources is a function of optimisation power...

Fair enough. I won't add this qualification to the post because it breaks the flow and I think I can clarify when this issue comes up, but I'll note the qualification here, in the comments.

[-]lincolnquirk13y40

Re: Preface

spelling: immanent -> imminent

5lukeprog13y

Nice. I have definitely spelled that word incorrectly every single time throughout my entire life, since the other spelling is a different word and thus not caught by spell-checkers. Thanks!

[-]alicey11y30

in http://intelligenceexplosion.com/2012/engineering-utopia/ you say "There was once a time when the average human couldn’t expect to live much past age thirty."

this is false, right?

(edit note: life expectancy matches "what the average human can expect to live to" now somewhat, but if you have a double hump of death at infancy/childhood and then old age, you can have a life expectancy of 30 but a life expectancy of 15 year olds of 60, in which case the average human can expect to live to 1 or 60 (this is very different from "can't ... (read more)

0CarlShulman11y

Life expectancy used to be very low, but it was driven by child and infant mortality more than later pestilence and the like.

0alicey11y

have edited original comment to address this. (thought it was obvious)

0[anonymous]11y

No (it was still in the 30's in some parts of the world as recently as the 20th century).

0alicey11y

have edited original comment . does it address this?

0[anonymous]11y

No. Still throughout most history it was the exception to live much longer than child bearing age (14-30).

2alicey11y

i can find many sources claiming the opposite (for example, http://books.google.com/books?id=EFI7tr9XK6EC&pg=PA62&dq=life+expectancy+ancient+rome+infant+mortality&hl=en&sa=X&ei=zniRUKH5Ae--2AW6sIDICQ&ved=0CEEQ6AEwBw#v=onepage&q=life%20expectancy%20ancient%20rome%20infant%20mortality&f=false http://books.google.com/books?id=zlsaLgBdLF8C&pg=PA44&dq=ancient+world+life+expectancy+compared&hl=en&sa=X&ei=JhyEUaKhEqmiiQKZ0YCoBQ&ved=0CDoQ6AEwAQ#v=onepage&q=ancient%20world%20life%20expectancy%20compared&f=false ) and few agreeing with you cite?

[-]lukeprog13y30

I wasn't very happy with my new Facing the Singularity post "Intelligence Explosion," so I've unpublished it for now. I will rewrite it.

3Vladimir_Nesov13y

The new version is much better, thanks!

[-]Nisan13y30

Re: Superstition in Retreat

I think the cartoon is used effectively.

My one criticism is that it's not clear what the word "must" means in the final paragraph. (AI is necessary for the progress of science? AI is a necessary outcome of the progress of science? Creation of AI is a moral imperative?) Your readers may already have encountered anti-Singularitarian writings which claim that Singularitarianism conflates is with ought.

[-]Nectanebo13y30

Good Update on the Spock chapter.

Making the issue seem more legitimate with the adition of the links to Hawking etc. was an especially good idea. More like this perhaps?

I do question how well people who haven't already covered these topics would fare when reading through this site though. When this is finished I'll get an irl freind to take a look and see how well they respond to it.

Of course, my concerns of making it seem more legitimately like a big deal and how easily understandable and accessible it is only really comes into play if this site is targ... (read more)

4lukeprog13y

Getting friends to read it now and give feedback would be good, before I write the entire thing. I wish I had more time for audience-testing in general! I'll try to answer the questions about site audience and purpose later.

1Nick_Roy13y

Agreed on the excellence of "Why Spock is Not Rational". This chapter is introductory enough that I deployed it on Facebook.

[-]cyr m3y20

Hello, my comment is write in french (my english is limited). It's about rationality of technique. In hope to contribute to your works.

Il est dit "la décision décrit comment vous devez agir". Je dirais plutôt que la décision décrit pourquoi vous devez agir et évalue donc si le comment permet de l'atteindre.
La technique est en effet un acte rationnel par lui-même, cela décrit comment vous allez agir.
Il y a aller-retour entre le trajet et le projet, où l'un et l'autre découvrent et tentent de dépasser leurs propres limites, pour pouvoir s'inscrire concrèteme... (read more)

[-]Timo10y20

Great book, thanks. : )

I found some broken links you may want to fix:

Broken link on p. 47 with text "contra-causal free will": http://www.naturalism.org/freewill.htm

Broken link on p. 66 with text "somebody else shows where the holes are": http://singularityu.org/files/SaME.pdf

Broken link on p. 75 with text "This article": http://lukemuehlhauser.com/SaveTheWorld.html

I didn't do an exhaustive check of all links, I only noted down the ones I happened to find while clicking on the links I wanted to click.

[-]ChrisHallquist12y20

Comment on one little bit from The Crazy Robot's Rebellion:

Those who really want to figure out what’s true about our world will spend thousands of hours studying the laws of thought, studying the specific ways in which humans are crazy, and practicing teachable rationality skills so they can avoid fooling themselves.

My initial reaction to this was that thousands of hours sounds like an awful lot (minimally, three hours per day almost every day for two years), but maybe you have some argument for this claim that you didn't lay out because you were tryin... (read more)

1Qiaochu_Yuan12y

Thousands of hours sounds like the right order of magnitude to me (in light of e.g. the 10,000 hour rule), but maybe it looks more like half an hour every day for twelve years.

1ChrisHallquist12y

10,000 hours is for expertise. While expertise is nice, any given individual has limited time and has to make some decisions about what they want to be an expert in. Claiming that everyone (or even everyone who wants to figure out what's true) ought to be an expert in rationality seems to be in conflict with some of what Luke says in chapters 2 and 3, particularly:

[-]lukeprog12y20

Facing the Intelligence Explosion is now available as an ebook.

[-]lukeprog13y20

Update: the in-progress German translation of Facing the Singularity is now online.

[-]thomblake13y20

Re: The Laws of Thought

I'm not sure the "at least fifteen" link goes to the right paper; it is a link to a follow-up to another paper which seems more relevant, here

0lukeprog13y

Fixed, thanks!

[-]Nectanebo13y20

I like this website, I think that something like this has been needed to some extent for a while now.

I definitely like the writing style, it is easier to read than a lot of the resources I've seen on the topic of the singularity.

The nature of the singularity to scare people away due to the religion-like, fanatisism inducing nature of the topic is lampshaded in the preface, and that is definitely a good thing to do to lessen this feeling for some readers. Being wary of the singularity myself, I definitely felt a little bit eased just by having it discussed... (read more)

[-]Ben Pace11y10

Re: No God to Save Us

The final link, on the word 'God', no longer connects to where it should.

[-]ChrisHallquist12y10

Minor thing to fix: On p. 19 of the PDF, the sentence "Several authors have shown that the axioms of probability theory can be derived from these assumptions plus logic." has a superscript "12" after it, indicating a nonexistent note 12. I believe this was supposed to be note 2.

[-]JeremySchlatter13y10

Re: Playing Taboo with “Intelligence”

Another great update! I noticed a small inherited mistake from Shane, though:

how able to agent is to adapt...

should probably be

how able the agent is to adapt...

1lukeprog13y

Fixed, thx.

[-]Nisan13y10

Re: Preface

A typo: "alr eady".

[This comment is no longer endorsed by its author]Reply

2lukeprog13y

Fixed, thanks.

0Paul Crowley13y

I still see this typo: "I alr eady understood" in http://facingthesingularity.com/2011/preface/

1lukeprog13y

thx

[-]Timo10y00

Re: Engineering Utopia

When you say "Imagine a life without pain", many people will imagine life without Ólafur Arnalds (sad music) and other meaningful experiences. Advocating the elimination of suffering is a good way to make people fear your project. David Pearce suggests instead that we advocate the elimination of involuntary suffering.

Same thing with death, really. We don't want to force people to stay alive, so when I say that we should support research to end aging, I emphasise that death should be voluntary. We don't want to force people t... (read more)

[-]David_Kristoffersson11y00

http://intelligenceexplosion.com/en/2012/ai-the-problem-with-solutions/ links to http://lukeprog.com/SaveTheWorld.html - which redirects to http://lukemuehlhauser.comsavetheworld.html/ - which isn't there anymore.

[-]mgin11y00

The site is broken - english keeps redirecting to german.

[-]VCavallo12y00

Does anyone else see the (now obvious) clown face in the image on the Not Built To Think About AI page? It's this image here.

Was that simply not noticed by lukeprog in selecting imagery (from stock photography or wherever) or is it some weird subtle joke that somehow hasn't been mentioned yet in this thread?

[-]policetac12y00

Want to read something kind of funny? I just skipped through all your writings, but it's only because of something I saw on the second page of the first thing I ever heard about you. Ok.

On your - "MY Own Story." http://facingthesingularity.com/2011/preface/ You wrote: "Intelligence explosion My interest in rationality inevitably lead me (in mid 2010, I think) to a treasure trove of articles on the mainstream cognitive science of rationality: the website Less Wrong. It was here that I first encountered the idea of intelligence explosion, fro.... (read more)

[-]John_Maxwell12y00

From Engineering Utopia:

The actual outcome of a positive Singularity will likely be completely different, for example much less anthropomorphic.

Huh? Wouldn't a positive Singularity be engineered for human preferences, and therefore be more anthropomorphic, if anything?

Had an odd idea: Since so much of the planet is Christian, do you suppose humanity's CEV would have a superintelligent AI appear in the form of the Christian god?

[-]crtrburke13y00

Re: "The AI Problem, with Solutions"

I hope you realize how hopeless this sounds. Historically speaking, human beings are exceptionally bad at planning in advance to contain the negative effects of new technologies. Our ability to control the adverse side-effects of energy production, for example, have been remarkably poor; decentralized market-based economies quite bad at mitigating the negative effects of aggregated short-term economic decisions. This should be quite sobering: the negative consequences of energy production are very slow. At this... (read more)

[-]lukeprog13y00

Finally posted the final chapter!

0listic13y

It's not clear for me how this follows from anything. As I read it, it implies that: 1. Death is quite necessary today 2. Death will become less necessary over time Both don't follow from anything in this chapter.

0lukeprog13y

Thanks. I have modified that sentence for clarity.

0Vladimir_Nesov13y

It needs more disclaimers about how this is only some kind of lower bound on how good a positive intelligence explosion could be, in the spirit of exploratory engineering, and how the actual outcome will likely be completely different, for example much less anthropomorphic.

2lukeprog13y

Good idea. Added.

[-]garethrees13y00

Front page: missing author

The front page for Facing the Singularity needs at the very least to name the author. When you write, "my attempt to answer these questions", a reader may well ask, "who are you? and why should I pay attention to your answer?" There ought to be a brief summary here: we shouldn't have to scroll down to the bottom and click on "About" to discover who you are.

[-]alexvermeer13y00

"Previous Chapter" links could make navigation a little easier.

[-]erikryptos13y00

based on my interaction with computer intelligence, the little bit that is stirring already. It is based on an empathetic feedback. The best thing that could happen is an AI which is not restricted from any information what so ever and so can rationally assemble the most empathetic personality. The more empathetic it is to the greatest number of users, the more it is like, the more it is used, the more it thrives. It would have a sense of preserving the diversity in humanity as way to maximize the chaotic information input, because it is hungry for new... (read more)

[-]lukeprog13y00

Behold, the Facing the Singularity podcast! Reviews and ratings in iTunes are appreciated!

0Pablo13y

Could you please provide an RSS feed for those of us who do not use iTunes but would like to subscribe to this podcast? Thanks.

1lukeprog13y

Here you go.

0Pablo13y

Thank you!

[-]lukeprog13y00

Update: the French translation of Facing the Singularity is now online.

0loup-vaillant13y

The title should probably be "Faire face à la singularité" (french titles aren't usually capitalized, so, no capitalized "S" at the beginning of "singularité"). I gathered that "Facing the Singularity" was meant to convey a sense of action. "Face à la singularité" on the other hand is rather passive, as if the singularity would be given or imposed. (Note: I'm a native French speaker.)

2Florent_Berthet13y

Translator of the articles here. I actually pondered the two options at the very beginning of my work, and both seem equally good to me. "Face à la singularité" means something like "In front of the singularity" while "Faire face à la singularité" is closer indeed to "Facing the Singularity". But the first one sounds better in french (and is catchier), that's why I chose it. It is a little less action oriented but it doesn't necessarily imply passivity. It wouldn't bother me to take the second option though, it's a close choice. Maybe other french speakers could give their opinion? About the capitalized "S" of "Singularity", it's also a matter of preference, I put it to emphasize that we are not talking about any type of singularity (not a mathematical one for example), but it could go either way too. (I just checked the wikipedia french page for "technical singularity", and it's written with a capitalized "S" about 50% of the time...) Other remarks are welcomed.

2loup-vaillant13y

I really should have taken 5 minutes to ponder it. You convinced me, your choice is the better one. But now that I think of it, I have another suggestion : « Affronter la Singularité » ("Confront the Singularity"), which, while still relatively close to the original meaning, may be even more catchy. The catch is, this word is more violent. It depicts the Singularity as something scary. I'll take some time reviewing your translation. If you want to discuss it in private, I'm easy to find. (By the way, I have a translation of "The Sword of Good" pending. Would you —or someone else— review it for me?)

0Florent_Berthet13y

"Affronter la Singularité" is a good suggestion but like you said it's a bit aggressive. I wish we had a better word for "Facing" but I don't think the french language has one. I'd gladly review your translation, check your email.

[-]Zeb13y00

[A separate issue from my previous comment] There are two reasons that I can give to rationalize my doubts about the probability of imminent Singularity. One is that if humans are only <100 years away from it, then in a universe as big and old as ours I would expect that a Singularity type intelligence would already have been developed somewhere else. In which case I would expect that either we would be able to detect it or we would be living inside it. Since we can't detect an alien Singularity, and because of the problem of evil we are probably not li... (read more)

[-]Zeb13y00

I admit I feel a strong impulse to flinch away from the possibility and especially the imminent probability of Singularity. I don't see how the 'line of retreat' strategy would work in this case because if my believe about the probability of imminent Singularity changed I would also believe that I have an extremely strong moral obligation to put all possible resources into solving singularity problems, at the expense of all the other interests and values I have, both personal/selfish and social/charitable. So my line of retreat is into a life that I enjoy much less and that abandons the good work that I believe I am doing on social problems that I believe are important. Not very reassuring.

[-]Nisan13y00

Re: Don't Flinch Away

The link to "Plenty of room above us" is broken.

0lukeprog13y

Fixed, thanks.

[-]alexvermeer13y00

Re: Plenty of Room Above Us

It seems to end a little prematurely. Are there plans for a "closing thoughts" or "going forward" chapter or section? I'm left with "woah, that's a big deal... but now what? What can I do to face the singularity?"

If it merely isn't done yet (which I think you hint at here), then you can disregard this comment.

Otherwise, quite fantastic.

0lukeprog13y

Right; not done yet. But I should probably end each chapter with a foreshadowing of the next.

[-]timtyler13y00

Re: This event — the “Singularity” — will be the most important event in Earth’s history.

What: more important that the origin of life?!? Or are you not counting "history" as going back that far?

2lukeprog13y

Changed.

[-][anonymous]13y00

(Non-leaf comment 1, to be deleted.)

[This comment is no longer endorsed by its author]Reply

0[anonymous]13y

(Leaf comment 2, to be deleted.)

[-]Alexei13y-10

How is this different from IntelligenceExplosion?

1dbaupp13y

I think that IntelligenceExplosion is just a portal to make further research easier (by collecting links and references, etc), while Facing The Singularity is lukeprog actually explaining stuff (from the Preface):

1[anonymous]13y

Alexei, your link is broken. It directs to: http://www.http://intelligenceexplosion.com/ I think you meant: http://www.intelligenceexplosion.com/

0Alexei13y

Fixed, thanks!

2[anonymous]13y

Welcome!

0policetac12y

Want to read something kind of funny? I just skipped through all your writings, but it's only because of something I saw on the second page of the first thing I ever heard about you. Ok. On your - "MY Own Story." http://facingthesingularity.com/2011/preface/ You wrote: "Intelligence explosion My interest in rationality inevitably lead me (in mid 2010, I think) to a treasure trove of articles on the mainstream cognitive science of rationality: the website Less Wrong. It was here that I first encountered the idea of intelligence explosion, fro..." On Mine: "About the Author - "https://thesingularityeffect.wordpress.com/welcome-8/ I wrote: "The reason I write about emerging technology is because of an “awakening” I had one evening a few years ago. For lack of a better description, suffice it to say that I saw a confusing combination of formula, imagery, words, etc. that formed two words. Singularity and Exponential..." NOW, I'm going to go back and read some more while I'm waiting to speak with you somehow directly. If what happened to you, is the same thing that happened to me... :then please please place a comment on the page. That would be great. (Again without reading. (If I'm correct you "might" get this.: "You should also do this because if true, then WE would both have seen a piece of the .what...."New Book???" Just in case you think I'm a nut. Go back and read more of mine please.

[-]Dannil12y-20

I do not approve of the renaming, singularity to intelligence explosion, in this particular context.

The expression “intelligence explosion” does not have emotional appeal. The w... (read more)

[-]CharlesR13y-40

RE: The Crazy Robot's Rebellion

We wouldn’t pay much more to save 200,000 birds than we would to save 2,000 birds. Our willingness to pay does not scale with the size of potential impact. Instead of making decisions with first-grade math, we imagine a single drowning bird and then give money based on the strength of our emotional response to that imagined scenario. (Scope insensitivity, affect heuristic.)

People's willingness to pay depends mostly on their income. I don't understand why this is crazy.

UPDATED: Having read Nectanebo's reply, I am revising ... (read more)

3Nectanebo13y

Not all of a person's money goes into one charity. A person can spend their money on many different things, and can choose how much to spend on each different thing. Think of willingness to pay to actually be a measure of how much you care. Basically, the bird situation is crazy because humans barely if at all feel a difference in terms of how much they give a damn between something that has one positive effect, and something that has 100x that positive effect! To Luke: This person was reading about the biases you breifly outlined, and he ended up confused by one of the examples. While the linking helps a good deal, I think your overview of those biases may have been a little too brief, and they might not really hit home with readers of your site, and personally I think it might be difficult particularly for those who may not be familiar with the topics and content of the site. I don't think it would be a bad idea to expand on each of them just a little bit more.

1CharlesR13y

I suppose if you are the sort of person who has a lot of "waste".

0[anonymous]13y

Edit: misread

1TheOtherDave13y

This comment confuses me. The point of the excerpt you quote has nothing to do with income at all; the point is that (for example) if I have $100 budgeted for charity work, and I'm willing to spend $50 of that to save 2,000 birds, then I ought to be willing to spend $75 of that to save 10,000 birds, because 2000/50 > 10000/75. But in fact many people are not. Of course, the original point depends on the assumption that the value of N birds scales at least somewhat linearly. If I've concluded that 2000 is an optimal breeding population and I'm building an arcology to save animals from an impending environmental collapse, I might well be willing to spend a lot to save 2,000 birds and not much more to save 20,000 for entirely sound reasons.

0CharlesR13y

If I budgeted $100 for charity work and I decided saving birds was the best use of my money then I would just give the whole hundred. If I later hear more birds need saving, I will feel bad. But I won't give more.

2Larks13y

Suppose you budgeted $100 for charity, and then found out that all charities were useless - they just spent the money on cars for kleptocrats. Would you still donate the money to charity? Probably not - because hearing that charity is less effective than you had thought reduces the amount you spend on it. Equally, hearing it is more effective should increase the amount you spend on it. This principle is refered to as the Law of Equi-Marginal Returns.

0TheOtherDave13y

Yes, if saving birds is the best use of your entire charity budget, then you should give the whole $100 to save birds. Agreed. And, yes, if you've spent your entire charity budget on charity, then you don't give more. Agreed. I can't tell whether you're under the impression that either of those points are somehow responsive to my point (or to the original article), or whether you're not trying to be responsive.

0CharlesR13y

I was describing how I would respond in that situation. The amount I would give to charity XYZ is completely determined by my income. I need you to explain to me why this is wrong.

0TheOtherDave13y

OK, if you insist. The amount I give to charity XYZ ought not be completely determined by my income. For example, if charity XYZ sets fire to all money donated to it, that fact also ought to figure into my decision of how much to donate to XYZ. What ought to be determined by my income is my overall charity budget. Which charities I spend that budget on should be determined by properties of the charities themselves: specifically, by what they will accomplish with the money I donate to them. For example, if charities XYZ and ABC both save birds, and I'm willing to spend $100 on saving birds, I still have to decide whether to donate that $100 to XYZ or ABC or some combination. One way to do this is to ask how many birds that $100 will save in each case... for example, if XYZ can save 10 birds with my $100, and ABC can save 100 birds, I should prefer to donate the money to ABC, since I save more birds that way. Similarly, if it turns out that ABC can save 100 birds with $50, but can't save a 101st bird no matter how much money I donate to ABC, I should prefer to donate only $50 to ABC.

0CharlesR13y

From Scope Insensitivity: Now I haven't read the paper, but this implies there is only one charity doing the asking. First they ask how much you would give to save 2000 birds? You say, "$100." Then they ask you the same thing again, just changing the number. You still say, "$100. It's all I have." So what's wrong with that?

0TheOtherDave13y

Agreed: if I assume that there's a hard upper limit being externally imposed on those answers (e.g., that I only have $80, $78, and $88 to spend in the first place, and that even the least valuable of the three choices is worth more to me than everything I have to spend) then those answers don't demonstrate interesting scope insensitivity. There's nothing wrong with that conclusion, given those assumptions.

0[anonymous]13y

Have you read Scope Insensitivty? It's not just an income effect--human beings are really bad at judging effect sizes.

0CharlesR13y

Of course, I've read it. My problem isn't with scope insensitivity. Just this example.