Making your explicit reasoning trustworthy

82 AnnaSalamon 29 October 2010 12:00AM

Or: “I don’t want to think about that!  I might be left with mistaken beliefs!”

Related to: Rationality as memetic immune disorder; Incremental progress and the valleyEgan's Law.

tl;dr: Many of us hesitate to trust explicit reasoning because... we haven’t built the skills that make such reasoning trustworthy.  Some simple strategies can help.

Most of us are afraid to think fully about certain subjects.

Sometimes, we avert our eyes for fear of unpleasant conclusions.  (“What if it’s my fault? What if I’m not good enough?”)

But other times, oddly enough, we avert our eyes for fear of inaccurate conclusions.[1]  People fear questioning their religion, lest they disbelieve and become damned.  People fear questioning their “don't walk alone at night” safety strategy, lest they venture into danger.  And I find I hesitate when pondering Pascal’s wager, infinite ethics, the Simulation argument, and whether I’m a Boltzmann brain... because I’m afraid of losing my bearings, and believing mistaken things.

Ostrich Theory, one might call it.  Or I’m Already Right theory.  The theory that we’re more likely to act sensibly if we don’t think further, than if we do.  Sometimes Ostrich Theories are unconsciously held; one just wordlessly backs away from certain thoughts.  Other times full or partial Ostrich Theories are put forth explicitly, as in Phil Goetz’s postthis LW comment, discussions of Tetlock's "foxes vs hedgehogs" research, enjoinders to use "outside views", enjoinders not to second-guess expert systems, and cautions for Christians against “clever arguments”.

continue reading »

Living in Many Worlds

18 Eliezer_Yudkowsky 05 June 2008 02:24AM

Followup toMany Worlds, One Best Guess

Some commenters have recently expressed disturbance at the thought of constantly splitting into zillions of other people, as is the straightforward and unavoidable prediction of quantum mechanics.

Others have confessed themselves unclear as to the implications of many-worlds for planning:  If you decide to buckle your seat belt in this world, does that increase the chance of another self unbuckling their seat belt?  Are you being selfish at their expense?

Just remember Egan's Law:  It all adds up to normality.

(After Greg Egan, in Quarantine.)

continue reading »