Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Affective Death Spirals

29 Post author: Eliezer_Yudkowsky 02 December 2007 04:44PM

Followup toThe Affect Heuristic, The Halo Effect

Many, many, many are the flaws in human reasoning which lead us to overestimate how well our beloved theory explains the facts.  The phlogiston theory of chemistry could explain just about anything, so long as it didn't have to predict it in advance.  And the more phenomena you use your favored theory to explain, the truer your favored theory seems—has it not been confirmed by these many observations?  As the theory seems truer, you will be more likely to question evidence that conflicts with it.  As the favored theory seems more general, you will seek to use it in more explanations.

If you know anyone who believes that Belgium secretly controls the US banking system, or that they can use an invisible blue spirit force to detect available parking spaces, that's probably how they got started.

(Just keep an eye out, and you'll observe much that seems to confirm this theory...)

This positive feedback cycle of credulity and confirmation is indeed fearsome, and responsible for much error, both in science and in everyday life.

But it's nothing compared to the death spiral that begins with a charge of positive affect—a thought that feels really good.

A new political system that can save the world.  A great leader, strong and noble and wise.  An amazing tonic that can cure upset stomachs and cancer.

Heck, why not go for all three?  A great cause needs a great leader.  A great leader should be able to brew up a magical tonic or two.

The halo effect is that any perceived positive characteristic (such as attractiveness or strength) increases perception of any other positive characteristic (such as intelligence or courage).  Even when it makes no sense, or less than no sense.

Positive characteristics enhance perception of every other positive characteristic?  That sounds a lot like how a fissioning uranium atom sends out neutrons that fission other uranium atoms.

Weak positive affect is subcritical; it doesn't spiral out of control.  An attractive person seems more honest, which, perhaps, makes them seem more attractive; but the effective neutron multiplication factor is less than 1.  Metaphorically speaking.  The resonance confuses things a little, but then dies out.

With intense positive affect attached to the Great Thingy, the resonance touches everywhere.  A believing Communist sees the wisdom of Marx in every hamburger bought at McDonalds; in every promotion they're denied that would have gone to them in a true worker's paradise; in every election that doesn't go to their taste, in every newspaper article "slanted in the wrong direction".  Every time they use the Great Idea to interpret another event, the Great Idea is confirmed all the more.  It feels better—positive reinforcement—and of course, when something feels good, that, alas, makes us want to believe it all the more.

When the Great Thingy feels good enough to make you seek out new opportunities to feel even better about the Great Thingy, applying it to interpret new events every day, the resonance of positive affect is like a chamber full of mousetraps loaded with ping-pong balls.

You could call it a "happy attractor", "overly positive feedback", a "praise locked loop", or "funpaper".  Personally I prefer the term "affective death spiral".

Coming tomorrow:  How to resist an affective death spiral.  (Hint:  It's not by refusing to ever admire anything again, nor by keeping the things you admire in safe little restricted magisteria.)

 

Part of the Death Spirals and the Cult Attractor subsequence of How To Actually Change Your Mind

Next post: "Resist the Happy Death Spiral"

Previous post: "Mere Messiahs"

Comments (37)

Sort By: Old
Comment author: JayDugger 02 December 2007 05:45:38PM 2 points [-]

Please define "magisteria" in the follow-up post. I tried three dictionaries without finding its definition.

Comment author: Ron_Hardin 02 December 2007 05:52:41PM 0 points [-]

Phlogiston was the cause of fire. It's a reification error, is all. Like ``power'' in political discourse, which is supposed to be a thing you can acquire, or lose, or contest. Whole analyses depend on it.

Comment author: Video2 02 December 2007 05:55:11PM 2 points [-]

magisteria (plural) – Realms of belief, for example, the realm of religious belief taken together with the realm of scientific belief. The absence of legitimate conflict between these realms was termed non-overlapping magisteria by Steven J. Gould.

Comment author: Recovering_irrationalist 02 December 2007 07:06:12PM 4 points [-]

(Just keep an eye out, and you'll observe much that seems to confirm this theory...)

I hope everyone was paying attention to that bit :-)

Coming tomorrow: How to resist an affective death spiral.

Please include judging how much to resist what may partly be a due to the spiral, so as not to overcompensate. Sometimes a "Great Thingy" is genuinely great.

Comment author: sam_kayley 02 December 2007 07:23:07PM 1 point [-]

Affective death spiral sounds like something to do with depression, praise locked loop may give a more accurate impression of the idea.

Comment author: Cyan2 02 December 2007 08:24:14PM 5 points [-]

"Affective death spiral" sounds like the process by which I became a militant evangelical Bayesian. But I got better: now I'm only a fundamentalist Bayesian, and my faith does not require me to witness the Bayesian Gospel to those who aren't interested.

Comment author: Joshua_Fox 02 December 2007 08:45:10PM 0 points [-]

I've always thought it was silly to call great football players "heroes." But in fact, people can be heroes (in the sense of role models) in one area of life and not in others. You can admire and be inspired by a role model's athleticism, intellectual honesty, kindness, etc. even though these are not usually found all together in one person.

Comment author: Cihan_Baran 02 December 2007 11:18:11PM 2 points [-]

This reminds me of a Karl Popper excerpt that I read several years ago. Popper levels similar charges against Marxism and Freudianism:

http://www.stephenjaygould.org/ctrl/popper_falsification.html

Thanks

Comment author: Ron_Hardin 02 December 2007 11:34:57PM 0 points [-]

Death spiral comes from airplanes and pilot disorientation leading to corrective action making a descending turn progressively worse. Without the disorientation, it doesn't happen.

Flying blind without instruments leads to disorientation very fast, if you're doing the flying. If you're just a passenger, you reorient from what the pilot does; but it's fatal if the pilot does that, without instruments to reorient himself from.

Disorientation is the key to take away.

Comment author: Eliezer_Yudkowsky 03 December 2007 12:29:53AM 1 point [-]

What would you think of "Happy Death Spiral"?

Comment author: Caledonian2 03 December 2007 01:41:09AM 8 points [-]

I would probably avoid taunting it.

Comment author: Tom_McCabe2 03 December 2007 02:39:06AM 0 points [-]

"Coming tomorrow: How to resist an affective death spiral. (Hint: It's not by refusing to ever admire anything again, nor by keeping the things you admire in safe little restricted magisteria.)"

Hmmm... maybe you could consider scenarios in which the Great Thingy gets you killed or seriously injured? Or by extrapolating it out until you reach predictions that are obviously absurd (eg, my boss is part of a government anti-Marxist conspiracy)?

Comment author: bigjeff5 11 February 2011 09:39:19PM 1 point [-]

Or by extrapolating it out until you reach predictions that are obviously absurd (eg, my boss is part of a government anti-Marxist conspiracy)?

That works just fine until your boss actually is part of a government anti-Marxist conspiracy...

Comment author: Stuart_Armstrong 03 December 2007 02:45:52PM 0 points [-]

Coming tomorrow: How to resist an affective death spiral.

Listening to some really good satire or mockery of the thing admired would help - it would dampen your emotional commitment to it, while leaving your rational commitment intact.

Trying to picture a world - sci-fi if needed - where your pet theory is not true may help, as long as you can create a reasonable functioning world, not a caricature...

But I'm feeling it's something far more cunning coming along...

Comment author: Stuart_Armstrong 03 December 2007 03:32:18PM 12 points [-]

On a more serious note: cut up your Great Thingy into smaller independent ideas, and treat them as independent.

For instance a marxist would cut up Marx's Great Thingy into a theory of value of labour, a theory of the political relations between classes, a theory of wages, a theory on the ultimate political state of mankind. Then each of them should be assessed independently, and the truth or falsity of one should not halo on the others. If we can do that, we should be safe from the spiral, as each theory is too narrow to start a spiral on its own.

Same thing for every other Great Thingy out there.

Comment author: g 03 December 2007 10:25:47PM 0 points [-]

But some Great Thingies might not be readily splittable. For instance, consider the whole edifice of theoretical physics, which is a pretty good candidate for a genuinely great Thingy (though not of quite the same type as most of the Great Thingies under discussion here). Each bit makes most sense in the context of the whole structure, and you can only appreciate why a given piece of evidence is evidence for one bit if you have all the other bits available to do the calculations with.

Of course, all this *could* just indicate that the whole edifice of theoretical physics (if taken as anything more than a black box for predicting observations) is a self-referential self-supporting delusion, and in a manner of speaking it's not unlikely that that's so -- i.e., the next major advance in theoretical physics could well overturn all the fundamentals while leaving the empirical consequences almost exactly the same. Be that as it may, much of the value of theoretical physics comes from the fact that it *is* a Great Thingy and not just a collection of Little Thingies, and it seems like it would be a shame to adopt a mode of thinking that prevents us appreciating it as such.

Comment author: Adirian 03 December 2007 11:25:24PM 1 point [-]

Notably, regarding theoretical physics, there are at least nine models for modern theoretical physics, all of which can perfectly explain the empirical observations, and all of which are completely and totally contradictory to one another. (Okay, almost all of which. There are a few compatibilities scattered amongst them. Neorealism can work fine with the multiverse model, and there are a small handful of models which are derived from Bohr's interpretations and are semicompatible with one another.)

Comment author: g 04 December 2007 12:09:03AM 4 points [-]

I think "completely and totally contradictory" is putting it too strongly, since they do in fact all agree about all observations we have ever been able to make or ever anticipate being able to make. Extreme verificationists would argue that the bits they disagree about are meaningless :-).

Comment author: Adirian 04 December 2007 12:16:53AM 1 point [-]

They agree about observations - but we already have the observations, so that doesn't mean much. Any theory worth thinking about isn't going to disagree about those observations, which, after all, they are created to explain. They disagree in every way it is meaningful that they, theories about the reason why, MAY disagree - in the reasons why. And extreme verificationists can go take a leap off a logical cliff when it comes to discussing differences in the reasons why something may be.

Comment author: Nick_Tarleton 04 December 2007 12:41:52AM 0 points [-]

"they do in fact all agree about all observations we have ever been able to make or ever anticipate being able to make."

Not entirely true.

Comment author: g 04 December 2007 12:58:39AM 0 points [-]

Nick: Oh, sorry, I forgot that there are still people who take the Copenhagen interpretation seriously. Though actually I suspect that they might just decree that observation by a *reversible* conscious observer doesn't count. That would hardly be less plausible than the Copenhagen interpretation itself. :-)

(I also have some doubt as to whether sufficiently faithful reversibility is feasible. It's not enough for the system to be restored to its prior state as far as macroscopic observations go; the reversal needs to be able to undo decoherence, so to speak. That seems like a very tall order.)

Adirian: the fact that their agreement-about-observations was predictable in advance doesn't make it any less an agreement. (And if you're talking only about the parts of those theories that are "theories about the reasons why", bracketing all the agreements about what's observed and how to calculate it, then I don't think you are entitled to call the things that disagree completely "models for modern theoretical physics".)

Comment author: Adirian 04 December 2007 01:12:39AM 0 points [-]

Nick - that proof works fine for any of the neorealist models, in which Everett's model is, variably, placed. The problem is in interpretation. Remember that there is great disagreement in the Copenhagen models about where, exactly, waveform collapse happens - after all, if one treats the quantum measurement device itself as being in a quantum state, then 100% correlation may be acceptable. (Because the waveform state of the computer wasn't collapsed until the first and third measurements were examined together.)

The real problem here is that the Copenhagen models are effectively unscientific, since it is fundamentally impossible to disprove the concept that anything that is unmeasured is in an uncertain/undefined state. It's an intellectual parlour trick, and shouldn't be taken seriously.

Comment author: bigjeff5 11 February 2011 10:56:12PM 0 points [-]

Upvoted because, while I don't know the details of the Copenhagen models, if it is true they rely on "the concept that anything that is unmeasured is in an uncertain/undefined state", then until some method of testing this state is devised the theories are effectively pseudo-science.

The Popper essay, originally mentioned above, describes the problem nicely.

It doesn't speak to the truth or untruth of the theory, just to its scientific status, or lack thereof. In a nutshell, if it's not testable, it's not scientific, whether it is true or not. This is why it should not be taken too seriously, at least not until it becomes testable.

Comment author: Adirian 04 December 2007 01:21:40AM 0 points [-]

the fact that their agreement-about-observations was predictable in advance doesn't make it any less an agreement. (And if you're talking only about the parts of those theories that are "theories about the reasons why", bracketing all the agreements about what's observed and how to calculate it, then I don't think you are entitled to call the things that disagree completely "models for modern theoretical physics".)

- It renders that agreement meaningless. If you curve-fit seven points, and come up with a thousand different formulas, the fact that each of these thousand formulas includes each of those seven points produces exactly no additional information. The fact of the matter is that we have discarded every formula which DIDN'T describe those points - that the remaining formulas do describe them tells us absolutely nothing about either the points or the relative value of the formulas with respect to one another.

At best, out of N formulas, each has a 1/N chance of being correct. (At worst, none of the formulas is correct.)

Comment author: Cyan2 04 December 2007 04:24:56PM 0 points [-]

At best, out of N formulas, each has a 1/N chance of being correct. (At worst, none of the formulas is correct.)

Technical note: Occam factors (and prior probabilities generally) can cause these chances to deviate from 1/N.

Comment author: Adirian 05 December 2007 01:04:21AM 0 points [-]

I didn't mean specifically, I meant on average. My apologies for the poor phrasing. Yes, any individual formula's odds of being correct can vary. (To deny this would be to deny Bayesian reasoning, and I think I might get mugged here if I tried that.)

Comment author: Acheman 12 December 2007 12:34:24PM 0 points [-]

Don't know if that'll solve matters, just trying. This does seem very Popperian - in a bad way, in that it's an oversimplified approach to theory-formation. What do you think about Kuhn, who finds this kind of reinforcement in normal, productive science - but still allows a distinction between evidence-based science and entirely circular nonscience? What about the idea that we have 'rings' of beliefs, and will sacrifice any number of 'outer-ring' theory detail to preserve our core beliefs?

Comment author: Acheman 12 December 2007 12:47:56PM 0 points [-]

Yeah, the 'help' was a futile attempt to close the open italics tag. Didn't work, obviously.

Comment author: g 12 December 2007 12:59:18PM 0 points [-]

Adirian (sorry for not noticing your response sooner), the situation is more like: we have a million data points and several models that all fit those points very precisely and all agree very precisely on how to interpolate between those points -- but if we try to use them to extrapolate wildly, into regions where in fact we have no way of getting any real data points, they diverge. It also turns out that within the region where we can actually get data -- where the models agree -- they don't agree merely by coincidence, but turn out to be mathematically equivalent to one another.

You are welcome to describe this situation by saying that the models "completely and totally contradictory", but I think that would be pretty eccentric.

(This is of course merely an analogy. I think the reality is even less favourable to your case.)

Comment author: Chip_Smith 17 January 2008 02:27:33PM 1 point [-]

ADS may be observed, most tragically, in the history of "facilitated communication."

http://www.cqc.state.ny.us/hottopics/fcwheel.htm

Comment author: Steve3 15 July 2008 08:19:26PM 4 points [-]

I personally prefer The Law of Fives: "ALL THINGS HAPPEN IN FIVES, OR ARE DIVISIBLE BY OR ARE MULTIPLES OF FIVE, OR ARE SOMEHOW DIRECTLY OR INDIRECTLY APPROPRIATE TO 5."

With the corollary: "I find the Law of Fives to be more and more manifest the harder I look."

cf. Foucault's Pendulum; the entire novel.

Comment author: Dojan 16 October 2011 01:04:38AM 0 points [-]

Ever since reading Hitchhikers Guide to the Galaxy I've seen the number 42 pop up at an alarming rate... Though I guess people use that number more than average for that very same reason. (I know I do!)

Comment author: Nightflow 03 February 2011 11:36:23AM 0 points [-]

Good article but why do you only talk about positive "death" spirals It would be just thr same with negative toughts added insecutity iy would be even harder to break out.

Comment author: mat33 06 October 2011 11:06:35AM 0 points [-]

In the 20-th century, Richard Feinmann did point out that there may be some problem with how we patch our phisics by cutting out the neigbourhoods. Nowdays we are pathing the General Relativity with the dark matter (it wasn't predicted, really) and even dark matter. It looks like we'll have to patch some "too fast neitrino in the matter" fenomenon.

I am not claiming this "patching" business something intristically right and beautifull. Never. We'll have to propose some new theories. But... before we'll have some better theory, to patch General Relativity seems just the thing to do. May be - the only thing to do, sorry.

An average scientist (if there is such a thing) isn't expected to propose something better, than General Relativity. Not really. So, even as we teach scientists, most of 'em wouldn't need to remember, that "patching" old theories isn't the right thing to do, in the long run. As they may do nothing about it. These with Nobel Prize ambition level would be wise to remember it, thought.

Comment author: Laoch 14 November 2013 12:25:34PM 0 points [-]

Is there such a thing as negative feeling death spirals, where say fear or mental illness keeps pushing you towards a terrible thought, concept or idea.