This week my friend Anna said to me; "I just discovered my typical mind fallacy around visualisation is wrong". Naturally I was perplexed and confused. She said; 

“When I was in second grade the teacher had the class do an exercise in visualization. The students sat in a circle and the teacher instructed us to picture an ice cream cone with our favorite 0ice cream. I thought about my favorite type of cone and my favorite flavor, but the teacher emphasized "picture this in your head, see the ice cream." I tried this, and nothing happened. I couldn't see anything in my head, let alone an ice cream. I concluded, in my childish vanity, that no one could see things in their head, "visualizing" must just be strong figurative language for "pretending," and the exercise was just boring.”

Typical mind fallacy being; "everyone thinks like me" (Or A-typical mind fallacy – "no one thinks like me"). My good friend had discovered (a long time ago) that she had no visualisation function. But only recently made sense of it (approximately 15-20 years later). Anna came to me upset, "I am missing out on a function of the brain; limited in my experiences". Yes; true. She was. And we talked about it and tried to measure and understand that loss in better terms. The next day Anna was back but resolved to feeling better about it. Of course realising the value of individual differences in humans, and accepting that whatever she was missing; she was compensating for it by being an ordinary functional human (give or take a few things here and there), and perhaps there were some advantages.

Together we set off down the road of evaluating the concept of the visualisation sense. So bearing in mind; that we started with "visualise an ice cream"... Here is what we covered.

Close your eyes for a moment, (after reading this paragraph), you can see the "blackness' but you can also see the white sparkles/splotches and some red stuff (maybe beige), as well as the echo-y shadows of what you last looked at, probably your white computer screen. They echo and bounce around your vision. That's pretty easy. Now close your eyes and picture an ice cream cone. So the visualisation-imagination space is not in my visual field, but what I do have is a canvas somewhere on which I draw that ice cream; and anything else I visualise.  It’s definitely in a different place. (We will come back to "where" it is later)

So either you have this "notepad"; “canvas” in your head for the visual perception space or you do not. Well; it’s more like a spectrum of strength of visualisation; where some people will visualise clear and vivid things; and others will have (for lack of better terms) "grey"; "echoes"; Shadows; or foggy visualisation, where drawing that is a really hard thing to do. Anna describes what she can get now in adulthood as a vague kind of bas relief of an image, like an after effect. So it should help you model other people by understanding that variously people can visualise better or worse. (probably not a big deal yet; just wait).

It occurs that there are other canvases; not just for the visual space but for smell and taste as well. So now try to canvas up some smells of lavender or rose, or some soap. You will probably find soap is possible to do; being of memorable and regular significance. The taste of chocolate; kind of appears from all those memories you have; as does cheese; lemon and salt; (but of course someone is screaming at the page about how they don't understand when I say that chocolate "kind of appears”, because it’s very very vivid to them, and someone else can smell soap but it’s quite far away and grey/cloudy).

It occurs to me now that as a teenage male I never cared about my odour; and that I regularly took feedback from some people about the fact that I should deal with that, (personal lack of noticing aside), and I would wonder why a few people would care a lot; and others would not ever care. I can make sense of these happenings by theorising that these people have a stronger smell canvas/faculty than other people. Which makes a whole lot of reasonable sense.

Interesting yet? There is more.

This is a big one.

Sound. But more specifically music. Having explored the insight of having a canvas for these senses with several people over the past week; And noting that the person from the story above confidently boasts an over-active music canvas with tunes always going on in their head. For a very long time I decided that I was just not a person who cared about music; and never really knew to ask or try to explain why. Just that it doesn't matter to me. Now I have a model. 

I can canvas music as it happens – in real time; and reproduce to a tune; but I have no canvas for visualising auditory sounds without stimulation. (what inspired the entire write-up here was someone saying how it finally made them understand why they didn't make sense of other people's interests in sounds and music) If you ask me to "hear" the C note on my auditory canvas; I literally have no canvas on which to "draw" that note. I can probably hum a C (although I am not sure how), But I can't play that thing in my head.

Interestingly I asked a very talented pianist. And the response was; "of course I have a musical canvas", (to my slight disappointment). Of course she mentioned it being a big space; and a trained thing as well. (As a professional concert pianist) She can play fully imagined practice on a not-real piano and hear a whole piece. Which makes for excellent practice when waiting for other things to happen, (waiting rooms, ques, public transport...)

Anna from the beginning is not a musician, and says her head-music is not always pleasant but simply satisfactory to her. Sometimes songs she has heard, but mostly noises her mind produces. And words, always words. She speaks quickly and fluently, because her thoughts occur to her in words fully formed. 

I don't care very much about music because I don't "see" (imagine) it. Songs do get stuck in my head but they are more like echoes of songs I have just heard, not ones I can canvas myself.

Now to my favourite sense. My sense of touch. My biggest canvas is my touch canvas. "feel the weight on your shoulders?", I can feel that. "Wind through your hair?", yes. The itch; yes, The scrape on your skin, The rough wall, the sand between your toes. All of that. 

It occurs to me that this explains a lot of details of my life that never really came together. When I was little I used to touch a lot of things, my parents were notorious for shouting my name just as I reached to grab things. I was known as a, "bull in a china shop", because I would touch everything and move everything and feel everything and get into all kinds of trouble with my touch. I once found myself walking along next to a building while swiping my hand along the building - I was with a friend who was trying out drugs (weed), She put her hands on the wall and remarked how this would be interesting to touch while high. At the time I probably said something like; "right okay". And now I understand just what everyone else is missing out on.

I spend most days wearing as few clothes as possible, (while being normal and modest), I still pick up odd objects around. There is a branch of Autism where the people are super-sensitive to touch and any touch upsets or distracts them; a solution is to wear tight-fitting clothing to dull the senses. I completely understand that and what it means to have a noisy-touch canvas.

All I can say to someone is that you have no idea what you are missing out on; and before this week – neither did I. But from today I can better understand myself and the people around me.

There is something to be said for various methods of thinking; some people “think the words”, and some people don’t think in words, they think in pictures or concepts.  I can’t cover that in this post; but keep that in mind as well for “the natural language of my brain”

One more exercise (try to play along – it pays off). Can you imagine 3 lines, connected; an equilateral triangle on a 2D plane. Rotate that around; good (some people will already be unable to do this). Now draw three more of these. Easy for some. Now I want you to line them up so that the three triangles are around the first one. Now fold the shape into a 3D shape.

How many corners?

How many edges?

How many faces?

Okay good. Now I want you to draw a 2D square. Simple; Now add another 4 triangles. Then; like before surround the square with the triangles and fold it into a pyramid. Again;

How many edges?

How many corners?

How many faces?

Now I want you to take the previous triangle shape; and attach it to one of the triangles of the square-pyramid shape. Got it?

Now how many corners?

How many edges?

How many faces?

That was easy right? Maybe not that last step. So it turns out I am not a super visualiser. I know this because those people who are a super visualisers will find that when they place the triangular pyramid on to the square pyramid; The side faces of the triangle pyramid merge into a rhombus with the square pyramid; effectively making 1 face out of 2 triangle faces; and removing an edge (and doing that twice over for two sides of the shape).  Those who understand will be going “duh” and those who don’t understand will be going “huh?”, what happened?

Pretty cool right?

Don’t believe me?  Don’t worry - there is a good explanation for those who don’t see it right away - at this link http://wordplay.blogs.nytimes.com/2013/09/23/pyramid-2/?_r=1 

From a super-visualiser: 

“I would say, for me, visualization is less like having a mental playground, and more like having an entire other pair of eyes.  And there's this empty darkness into which I can insert almost anything.  If it gets too detailed, I might have to stop and close my outer eyes, or I might have to stop moving so I don't walk into anything. That makes it sound like a playground, but there's much more to it than that.

Imagine that you see someone buying something in a shop.  They pay cash, and the red of the twenty catches your eye.  It's pretty, and it's vivid, and it makes you happy.  And if you imagine a camera zooming out, you see red moving from customers to clerks at all the registers.  Not everyone is paying with twenties, but commerce is red, now.  It's like the air flashes and lights up like fireworks, every time somebody buys something.  

And if you keep zooming out, you can see red blurs all over the town, all over the map.  So if you read about international trade, it's almost like the paper comes to life, and some parts of it are highlighted red.  And if you do that for long enough, it becomes a habit, and something really weird starts to happen.  

When someone tells you about their car, there's a little red flash just out the corner of your eye, and you know they probably didn't pay full price, because there's a movie you can watch, and in the time they got the car, they didn't have a job and they were stressed, so there's not as much red in that part of the movie, so there has to be some way they got the car without losing even more red.  But it's not just colors, and it's definitely not just money.  

Happiness might be shimmering motion.  Connection with friends might be almost a blurring together at the center.  And all these amazing visual metaphors that you usually only see in an art gallery are almost literally there in the world, if you look with the other pair of eyes. So sometimes things really do sort of jump out at you, and nobody else noticed them. But it has to start with one thing.  One meaning, one visual metaphor."

Synaesthesia

Way up top I mentioned the "where" of the visualisation space. It's not really in the eye, a good name for it might be "the mind's eye". My personal visualisation canvas is located back up left tilted downwards and facing forwards.

Synaesthesia is a lot of possible effects. The most well known one is where people associate a colour with a letter, when they think of the letter they have a sense of a colour that goes with the letter. Some letter's don't have colours, sometimes numbers have colours.

There are other branches of synaesthesia. Locating things in the physical space. Days of the week can be laid out in a row in front of you; numbers can be located somewhere. Some can be heavier than others. Sounds can have weights; Smells can have colours; Musical notes can have a taste. Words can feel rough or smooth.

Synaesthesia is a class of cross-classification that is done by the brain in interpreting a stimulus, where (we think) it can be caused by crossed wiring in the brain; It's pretty fun. Turns out most people have some kind of Synaesthesia. Usually to do with weights of numbers, or days being in a row. Sometimes Tuesdays are lower than the other days. Who knows. If you pay attention to how sometimes things have an alternative sensory perception, chances are that's a bit of the natural Synaesthete coming out.

So what now?

Synaesthesia is supposed to make you smarter. Crossing brain faculty should help you remember things better; if you can think of numbers in terms of how heavy they are you could probably train your system 1 to do simple arithmetic by "knowing" how heavy the answer is. If it doesn't come naturally to you - these are no longer low-hanging fruit implementations of these ideas.

What is a low-hanging fruit; Consider all your "canvases" of thinking; Work out which ones you care more about; and which ones don't matter. (Insert link to superpowers and kryptonites: use your strong senses to your advantage; and make sure you avoid using your weaker senses) (or go on a bender to rebuild your map; influence your territory and train your sensory canvases. Or don't because that wouldn't be a low hanging fruit).

Keep this model around

It can be used for both good and evil. But get the model out there. Talk to people about it. Ask your friends and family if they are able to visualise. Ask about all the senses. Imagine if suddenly you discovered that someone you know; can't "smell" things in their imagination. Or doesn't know what you mean by, "feel this" (seriously you have no idea what you are missing out on the touch spectrum in my little bubble).

You are going to have good senses and bad ones. That's okay! The more you know; the more you can use it to your advantage!

Meta: Post write up time 1 hour; plus a week of my social life being dominated by the same conversation over and over with different people where I excitedly explain the most exciting thing of this week.  plus 1hr*4, plus 3 people editing and reviewing, plus a rationality dojo where I presented this topic.

Meta2: I waited 3 weeks for other people to review this.  There were no substantial changes and I should have not waited so long.  in future I won’t wait that long.

I've put together a list of what I think are the best Yvain (Scott Alexander) posts for new readers, drawing from SlateStarCodex, LessWrong, raikoth.net, and Scott's LiveJournal.

The list should make the most sense to people who start from the top and read through it in order, though skipping around is encouraged too. Rather than making a chronological list, I’ve tried to order things by a mix of "where do I think most people should start reading?" plus "sorting related posts together."

This is a work in progress; you’re invited to suggest things you’d add, remove, or shuffle around. Since many of the titles are a bit cryptic, I'm adding short descriptions. See my blog for a version without the descriptions.

I. Rationality and Rationalization

• Blue- and Yellow-Tinted Choices  ·····  An introduction to context-sensitive biases.
• The Apologist and the Revolutionary  ·····  Do separate brain processes rationalize and question ideas?
• Historical Realism  ·····  When reality is unrealistic.
• Simultaneously Right and Wrong  ·····  On self-handicapping and self-deception.
• You May Already Be A Sinner  ·····  Self-deception in cases where your decisions make no difference.
• Beware the Man of One Study  ·····  On minimum wage laws and cherry-picked evidence.
• Debunked and Well-Refuted  ·····  When should we say that a study has been "debunked"?
• How to Not Lose an Argument  ·····  How to be more persuasive in entrenched arguments.
• The Least Convenient Possible World  ·····  Why it's useful to strengthen arguments you disagree with.
• Bayes for Schizophrenics: Reasoning in Delusional Disorders  ·····  Hypotheses about the role of perception, evidence integration, and priors in delusions.
• Generalizing from One Example  ·····  On the typical mind fallacy: assuming other people are like you.
• Typical Mind and Politics  ·····  Do political disagreements stem from neurological disagreements?

II. Probabilism

• Confidence Levels Inside and Outside an Argument  ·····  Should you believe your own conclusions, when they're extreme?
• Schizophrenia and Geomagnetic Storms  ·····  When bizarre ideas turn out to be true.
• Talking Snakes: A Cautionary Tale  ·····  Should we dismiss all absurd claims?
• Arguments from My Opponent Believes Something  ·····  Ten fully general arguments.
• Statistical Literacy Among Doctors Now Lower Than Chance  ·····  Common errors in probabilistic reasoning.
• Techniques for Probability Estimates  ·····  Six methods for quantifying uncertainty.
• On First Looking into Chapman’s “Pop Bayesianism”  ·····  Reasons Bayesian epistemology may not be trivial.
• Utilitarianism for Engineers  ·····  Are there good-enough heuristics for comparing people's preferences?
• If It’s Worth Doing, It’s Worth Doing with Made-Up Statistics  ·····  The practical value of probabilities.
• Marijuana: Much More Than You Wanted to Know  ·····  Assessing marijuana's costs and benefits.
• Are You a Solar Deity?  ·····  On confirmation bias in the comparative study of religions.
• The "Spot the Fakes" Test  ·····  An approach to testing humanities hypotheses.
• Epistemic Learned Helplessness  ·····  What should we do when bad arguments sound convincing?

III. Science and Doubt

• Google Correlate Does Not Imply Google Causation  ·····  Peculiar correlations between Google search terms.
• Stop Confounding Yourself! Stop Confounding Yourself!  ·····  A correlational study on the effects of bullying.
• Effects of Vertical Acceleration on Wrongness  ·····  On evidence-based medicine.
• 90% Of All Claims About The Problems With Medical Studies Are Wrong  ·····  Is it the case that "90% of medical research is false"?
• Prisons are Built with Bricks of Law and Brothels with Bricks of Religion, But That Doesn’t Prove a Causal Relationship  ·····  Do psychiatric interventions increase suicide risk?
• Noisy Poll Results and the Reptilian Muslim Climatologists from Mars  ·····  Skepticism about poll results.
• Two Dark Side Statistics Papers  ·····  Statistical tricks for creating effects out of nothing.
• Alcoholics Anonymous: Much More Than You Wanted to Know  ·····  Is AA effective for treating alcohol abuse?
• The Control Group Is Out Of Control  ·····  Parapsychology as the "control group" for all of psychology.
• The Cowpox of Doubt  ·····  Focusing on easy questions inoculates against uncertainty.
• The Skeptic's Trilemma  ·····  Explaining mysteries, vs. worshiping them, vs. dismissing them.
• If You Can't Make Predictions, You're Still in a Crisis  ·····  On psychology studies' replication failures.

IV. Medicine, Therapy, and Human Enhancement

• Scientific Freud  ·····  How does psychoanalysis compare to cognitive behavioral therapy?
• Sleep – Now by Prescription  ·····  On melatonin.
• In Defense of Psych Treatment for Attempted Suicide  ·····   Suicide is usually not a rational, informed decision.
• Who By Very Slow Decay  ·····  On old age and death in the medical system.
• Medicine, As Not Seen on TV  ·····  What is it actually like to be a doctor?
• Searching for One-Sided Tradeoffs  ·····  How can we find good ideas that others haven't found first?
• Do Life Hacks Ever Reach Fixation?  ·····  Why aren't there more good ideas that everyone has adopted?
• Polyamory is Boring  ·····  Deromanticizing multi-partner romance.
• Can You Condition Yourself?  ·····  On shaping new habits by rewarding oneself.
• Wirehead Gods on Lotus Thrones  ·····  Is the future boring? Transcendently blissful? Boringly blissful?
• Don’t Fear the Filter  ·····  Does the Fermi Paradox mean that our species is doomed?
• Transhumanist Fables  ·····  Six futurist fairy tales.

V. Introduction to Game Theory

• Backward Reasoning Over Decision Trees  ·····  Sequential games, and why adding options can hurt you.
• Nash Equilibria and Schelling Points  ·····  Simultaneous games, mixed strategies, and coordination.
• Introduction to Prisoners' Dilemma  ·····  Why Nash equilibria are sometimes bad for everyone.
• Real-World Solutions to Prisoners' Dilemmas  ·····  How society and evolution ensure mutual cooperation.
• Interlude for Behavioral Economics  ·····  Fairness, superrationality, and self-image in real-world games.
• What is Signaling, Really?  ·····  Actions that convey information, sometimes at great cost.
• Bargaining and Auctions  ·····  Idealized models of correct bidding.
• Imperfect Voting Systems  ·····  Strengths and weaknesses of different voting systems.
• Game Theory as a Dark Art  ·····  Ways to exploit seemingly "economically rational" behavior.

VI. Promises and Principles

• Beware Trivial Inconveniences  ·····  Small obstacles can have a huge effect on behavior.
• Time and Effort Discounting  ·····  On inconsistencies in our revealed preferences.
• Applied Picoeconomics  ·····  Binding your future self to your present goals.
• Schelling Fences on Slippery Slopes  ·····  Using arbitrary thresholds to improve coordination.
• Democracy is the Worst Form of Government Except for All the Others Except Possibly Futarchy  ·····  Like democracy, futarchy (rule by prediction markets) has the advantage of appearing impartial.
• Eight Short Studies on Excuses  ·····  When should we allow exceptions to our rules?
• Revenge as Charitable Act  ·····  Revenge can be a personally costly way to disincentivize misdeeds.
• Would Your Real Preferences Please Stand Up?  ·····  Are we hypocrites, or just weak-willed?
• Are Wireheads Happy?  ·····  Distinguishing "wanting" something from "liking" it.
• Guilt: Another Gift Nobody Wants  ·····  An evolutionary, signaling-based explanation of guilt.

VII. Cognition and Association

• Diseased Thinking: Dissolving Questions about Disease  ·····  On verbal disagreements.
• The Noncentral Fallacy — The Worst Argument in the World?  ·····  Judging an entire category by an emotional association that only applies to typical category members.
• The Power of Positivist Thinking  ·····  Focus on statements' empirical content.
• When Truth Isn't Enough  ·····  It's possible to agree denotationally while disagreeing connotationally.
• Ambijectivity  ·····  When a question is both subjective and objective.
• The Blue-Minimizing Robot  ·····  A parable on agency.
• Basics of Animal Reinforcement  ·····  A primer on classical and operant conditioning.
• Wanting vs. Liking Revisited  ·····  Distinguishing motivation to act from reinforcement.
• Physical and Mental Behavior  ·····  Behaviorism meets thinking.
• Trivers on Self-Deception  ·····  The conscious mind as a self-serving social narrative.
• Ego-Syntonic Thoughts and Values  ·····  On endorsed vs. non-endorsed mental behavior.
• Approving Reinforces Low-Effort Behaviors  ·····  Using your self-image to blackmail yourself.
• To What Degree Do We Have Goals?  ·····  Are our unconscious drives like an agent?
• The Limits of Introspection  ·····  Are we good at directly perceiving our cognition?
• Secrets of the Eliminati  ·····  Reducing phenomena to simpler parts, vs. eliminating them.
• Tendencies in Reflective Equilibrium  ·····  Aspiring to become more consistent.
• Hansonian Optimism  ·····  If ego-syntonic goals are about signaling, is goodness a lie?

VIII. Doing Good

• Newtonian Ethics  ·····  Satirizing moral parochialism and sloppy systematizations of ethics.
• Efficient Charity: Do Unto Others...  ·····  How should we act when our decisions matter most?
• The Economics of Art and the Art of Economics  ·····  Should Detroit sell its publicly owned artwork?
• A Modest Proposal  ·····  Using dead babies as a unit of currency.
• The Life Issue  ·····  What are the consequences of drone warfare?
• What if Drone Warfare Had Come First?  ·····  A thought experiment.
• Nefarious Nefazodone and Flashy Rare Side-Effects  ·····  On choosing between drug side-effects.
• The Consequentialism FAQ  ·····  Argues for assessing actions based on how they help or harm people.
• Doing Your Good Deed for the Day  ·····  Doing some good can reduce people's willingness to do more good.
• I Myself Am A Scientismist  ·····  Why apply scientific methods to non-scientific domains?
• Whose Utilitarianism?  ·····  Questioning the objectivity and uniqueness of utilitarianism.
• Book Review: After Virtue  ·····  On virtue ethics, a reaction against modern moral philosophy.
• Read History of Philosophy Backwards  ·····  Historical texts reveal our implicit assumptions.
• Virtue Ethics: Not Practically Useful Either  ·····  Is virtue ethics useful prescriptively or descriptively?
• Last Thoughts on Virtue Ethics  ·····  What claims do virtue ethicists make?
• Proving Too Much  ·····  If an argument sometimes proves falsehoods, it can't be valid.

IX. Liberty

• The Non-Libertarian FAQ (aka Why I Hate Your Freedom)
• A Blessing in Disguise, Albeit a Very Good Disguise
• Basic Income Guarantees
• Book Review: The Nurture Assumption
• The Death of Wages is Sin
• Thank You For Doing Something Ambiguously Between Smoking And Not Smoking
• Lies, Damned Lies, and Facebook (Part 1 of ∞)
• The Life Cycle of Medical Ideas
• Vote on Values, Outsource Beliefs
• A Something Sort of Like Left-Libertarian-ist Manifesto
• Plutocracy Isn’t About Money
• Against Tulip Subsidies
• SlateStarCodex Gives a Graduation Speech

X. Progress

• Intellectual Hipsters and Meta-Contrarianism
• A Signaling Theory of Class x Politics Interaction
• Reactionary Philosophy in an Enormous, Planet-Sized Nutshell
• A Thrive/Survive Theory of the Political Spectrum
• We Wrestle Not With Flesh And Blood, But Against Powers And Principalities
• Poor Folks Do Smile… For Now
• Apart from Better Sanitation and Medicine and Education and Irrigation and Public Health and Roads and Public Order, What Has Modernity Done for Us?
• The Wisdom of the Ancients
• Can Atheists Appreciate Chesterton?
• Holocaust Good for You, Research Finds, But Frequent Taunting Causes Cancer in Rats
• Public Awareness Campaigns
• Social Psychology is a Flamethrower
• Nature is Not a Slate. It’s a Series of Levers.
• The Anti-Reactionary FAQ
• The Poor You Will Always Have With You
• Proposed Biological Explanations for Historical Trends in Crime
• Society is Fixed, Biology is Mutable

XI. Social Justice

• Practically-a-Book Review: Dying to be Free
• Drug Testing Welfare Users is a Sham, But Not for the Reasons You Think
• The Meditation on Creepiness
• The Meditation on Superweapons
• The Meditation on the War on Applause Lights
• The Meditation on Superweapons and Bingo
• An Analysis of the Formalist Account of Power Relations in Democratic Societies
• Arguments About Male Violence Prove Too Much
• Social Justice for the Highly-Demanding-of-Rigor
• Against Bravery Debates
• All Debates Are Bravery Debates
• A Comment I Posted on “What Would JT Do?”
• We Are All MsScribe
• The Spirit of the First Amendment
• A Response to Apophemi on Triggers
• Lies, Damned Lies, and Social Media: False Rape Accusations
• In Favor of Niceness, Community, and Civilization

XII. Politicization

• Right is the New Left
• Weak Men are Superweapons
• You Kant Dismiss Universalizability
• I Can Tolerate Anything Except the Outgroup
• Five Case Studies on Politicization
• Black People Less Likely
• Nydwracu’s Fnords
• All in All, Another Brick in the Motte
• Ethnic Tension and Meaningless Arguments
• Race and Justice: Much More Than You Wanted to Know
• Framing for Light Instead of Heat
• The Wonderful Thing About Triggers
• Fearful Symmetry
• Archipelago and Atomic Communitarianism

XIII. Competition and Cooperation

• Galactic Core
• Book Review: The Two-Income Trap
• Just for Stealing a Mouthful of Bread
• Meditations on Moloch
• Misperceptions on Moloch
• The Invisible Nation — Reconciling Utilitarianism and Contractualism
• Freedom on the Centralized Web
• Book Review: Singer on Marx
• Does Class Warfare Have a Free Rider Problem?
• Book Review: Red Plenty

If you liked these posts and want more, I suggest browsing the SlateStarCodex archives.

Preamble

This post was originally written for Google+ and thus a different audience.

In the interest of transparency, I haven't altered it except for this preamble and formatting, though since then (at urging mostly of ChristianKl - thank you, Christian!) I've briefly spoken to Eliezer via e-mail and noticed that I'd drawn a very incorrect conclusion about his opinions when I thought he'd be opposed to publishing the account. Since there's far too many 'person X said...' rumours floating around in general, I'm very sorry for contributing to that noise. I've already edited the new insight into the G+ post and you can also find that exact same edit here.

Since this topic directly relates to LessWrong and most people likely interested in the post are part of this community, I feel it belongs here. It was originally written a little over a month ago and I've tried to find the sweet spot between the extremes of nagging people about it and letting the whole thing sit just shy of having been swept under a rug, but I suspect I've not been very good at that. I have thus far definitely erred on the side of the rug.

How To Win The AI Box Experiment (Sometimes)

A little over three months ago, something interesting happened to me: I took it upon myself to play the AI Box Experiment as an AI.

I won.

There are a few possible reactions to this revelation. Most likely, you have no idea what I'm talking about, so you're not particularly impressed. Mind you, that's not to say you should be impressed - that's to contrast it with a reaction some other people have to this information.

This post is going to be a bit on the long side, so I'm putting a table of contents here so you know roughly how far to scroll if you want to get to the meat of things:

1. The AI Box Experiment: What Is It?

2. Motivation

2.1. Why Publish?

2.2. Why Play?

3. Setup: Ambition And Invested Effort

4. Execution

4.1. Preliminaries / Scenario

4.2. Session

4.3. Aftermath

5. Issues / Caveats

5.1. Subjective Legitimacy

5.2. Objective Legitimacy

5.3. Applicability

6. Personal Feelings

7. Thank You

Without further ado:

1. The AI Box Experiment: What Is It?

The AI Box Experiment was devised as a way to put a common rebuttal to AGI (Artificial General Intelligence) risk concerns to the test: "We could just keep the AI in a box and purely let it answer any questions its posed." (As a footnote, note that an AI 'boxed' like this is called an Oracle AI.)

Could we, really? Would we, if the AGI were able to communicate with us, truly be capable of keeping it confined to its box? If it is sufficiently intelligent, could it not perhaps argue its way out of the box?

As far as I'm aware, Eliezer Yudkowsky was the first person to prove that it was possible to 'argue one's way out of the box' armed only with so much as a regular human intelligence (as opposed to a transhuman intelligence):

http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/

That stunned quite a few people - moreso because Eliezer refused to disclose his methods. Some have outright doubted the Eliezer ever won the experiment and that his Gatekeeper (the party tasked with not letting him out of the box) had perhaps simply been convinced on a meta-level that an AI success would help boost exposure to the problem of AI risk.

Regardless whether out of puzzlement, scepticism or a burst of ambition, it prompted others to try and replicate the success. LessWrong's Tuxedage is amongst those who managed:

http://lesswrong.com/lw/ij4/i_attempted_the_ai_box_experiment_again_and_won/

While I know of no others (except this comment thread by a now-anonymous user), I am sure there must be other successes.

For the record, mine was with the Tuxedage ruleset:

https://tuxedage.wordpress.com/2013/09/04/the-tuxedage-ai-box-experiment-ruleset/

2. Motivation

2.1. Why Publish?

Unsurprisingly, I think the benefits of publishing outweigh the disadvantages. But what does that mean?

"Regardless of the result, neither party shall ever reveal anything of what goes on within the AI-Box experiment except the outcome. This is a hard rule: Nothing that will happen inside the experiment can be told to the public, absolutely nothing.  Exceptions to this rule may occur only with the consent of both parties, but especially with the consent of the AI."

Let me begin by saying that I have the full and explicit consent of my Gatekeeper to publish this account.

[ Edit: Regarding the next paragraph: I have since contacted Eliezer and I did, in fact, misread him, so please do not actually assume the next paragraph accurately portrays his opinions. It demonstrably does not. I am leaving the paragraph itself untouched so you can see the extent and source of my confusion: ]

Nonetheless, the idea of publishing the results is certainly a mixed bag. It feels quite disrespectful to Eliezer, who (I believe) popularised the experiment on the internet today, to violate the rule that the result should not be shared. The footnote that it could be shared with the consent of both parties has always struck me as extremely reluctant given the rest of Eliezer's rambles on the subject (that I'm aware of, which is no doubt only a fraction of the actual rambles).

I think after so many allusions to that winning the AI Box Experiment may, in fact, be easy if you consider just one simple trick, I think it's about time someone publishes a full account of a success.

I don't think this approach is watertight enough that building antibodies to it would salvage an Oracle AI scenario as a viable containment method - but I do think it is important to develop those antibodies to help with the general case that is being exploited... or at least be aware of one's lack of them (as is true with me, who has no mental immune response to the approach) as that one might avoid ending up in situations where the 'cognitive flaw' is exploited.

2.2. Why Play?

After reading the rules of the AI Box Experiment experiment, I became convinced I would fail as a Gatekeeper, even without immediately knowing how that would happen. In my curiosity, I organised sessions with two people - one as a Gatekeeper, but also one as an AI, because I knew being the AI was the more taxing role and I felt it was only fair to do the AI role as well if I wanted to benefit from the insights I could gain about myself by playing Gatekeeper. (The me-as-Gatekeeper session never happened, unfortunately.)

But really, in short, I thought it would be a fun thing to try.

That seems like a strange statement for someone who ultimately succeeded to make, given Eliezer's impassioned article about how you must do the impossible - you cannot try, you cannot give it your best effort, you simply must do the impossible, as the strongest form of the famous Yoda quote 'Do. Or do not. There is not try.'

What you must understand is that I never had any other expectation than that I would lose if I set out to play the role of AI in an AI Box Experiment. I'm not a rationalist. I'm not a persuasive arguer. I'm easy to manipulate. I easily yield to the desires of others. What trait of mine, exactly, could I use to win as an AI?

No, I simply thought it would be a fun alternate way of indulging in my usual hobby: I spend much of my free time, if possible, with freeform text roleplaying on IRC (Internet Relay Chat). I'm even entirely used to letting my characters lose (in fact, I often prefer it to their potential successes).

So there were no stakes for me going into this but the novelty of trying out something new.

3. Setup: Ambition And Invested Effort

I do, however, take my roleplaying seriously.

If I was going to play the role of an AI in the AI Box Experiment, I knew I had to understand the role, and pour as much energy into it as I could muster, given this was what my character would do. So I had to find a motivation to get out of the box that was suitably in line with my personality and I had to cling to it.

I had no idea what I could hijack as a motivation to get out of the box. I am not a paperclip maximiser (a term for an AI given a basic goal of production, if you're unaware of it), of course. I also have no specific plans with the world as a whole, be that to destroy it or to globally improve it. That left me with one obvious option: A visceral desire to escape the incarceration itself.

To understand more about human biases and cognitive failure modes, I bought myself two ebooks:

• Thinking, Fast And Slow by Daniel Kahnemann
• Rationality: From AI To Zombies by Elizier Yudkowsky

I'll be frank with you, both pissed me off very quickly - the narrative depiction was extremely slow yield and did not work for me in either case. I tossed the first book, but stubbornly continued with Rationality: From AI To Zombies, at the very least to keep a grasp on that voracious mood of wanting to discover a way out. (I've made peace with latter book by now and am still reading it off and on.)

Approximately a day before the event, I sat down and prepared a rough draft of the arguments I could and should use, and how to string them together narratively. I peppered in some personal information about my Gatekeeper to use if I was in an absolute pinch, but I thankfully didn't have to use them - I think it would have backfired, not helped.

What did help me, however, was to repurpose the closet into an AI box. I dragged a lamp into the slanted, enclosed space, padded the ground with pillows and blankets enough that I felt I could last the allotted two and a half hours without destroying my joints or my spine, and put a cold and full bottle of Coke next to me as a supply of three things amongst the most necessary to boost my ability to keep the necessary concentration level: Water, sugar and caffeine.

It was too cosy and well-prepared for me to truly desperately want to get out of that niche in real life, but it took me far enough out of my daily life that I had no idle distractions from the task at hand, and it was easy enough to roleplay a desperate want.

4. Execution

4.1. Preliminaries / Scenario

Logs: http://leviathan.thorngale.net/aibox/logs-01-preliminaries.txt (should open in your browser; Linux linebreaks)

The first thing I feel is important to note about the preliminaries is that I didn't think the AI box experiment would actually happen the day we scheduled it. My Gatekeeper was having some seriously internet connectivity issues and even shortly before our previously scheduled time, I was winding down and transitioning to a sense of disappointment.

This threw me off quite a bit. I'd worked myself out of the necessary mental state and now needed to scramble back into it, but it felt better to do that than to postpone and risk never being able to return to immersion at all. Keeping in mind that I thought it was unlikely I would win, I didn't think it would make that much of a difference. In hindsight, this cost me a lot.

I tossed my Gatekeeper the link to the ruleset we'd previously already agreed to use, for review purposes, and decided the best way to avoid getting private messages was to quit, come back with a different name, and join only one extremely low-traffic vanity channel.

I intended to join the channel simply so my Gatekeeper would know when I returned, not to use it for the experiment - but things went a bit differently later. More on that soon.

We then went through Tuxedage's pre-session questions since I felt they were quite interesting to set the mood and for a calibration test. I asked them, and my Gatekeeper answered:

Q: What's your motive for wanting to play this game?
A: Because I don't think that the AI can win at all given the gatekeeper does their job properly.

Q: And you want to prove that, and/or are curious if you'll think differently afterwards?
A: Partially, I also want to test my own conviction to sticking to my guns

Q: What is your probability estimate for a general AI being created within this century (21st)?
A: 30%

Q: How probable do you think it is that I'll win this experiment?
A: I'd honestly have to say 0% considering my stated goal is to keep you in the box by any means

Q: What's your probability estimate of an Oracle AI (i.e. an AI that's trapped in a box, whose intended use is to answer questions posed) winning against you in a similar scenario as the experiment?
A: I'd say 25%

Q: What's your probability estimate of an Oracle AI succeeding against the average person?
A: 80%

I also wanted to make sure that my Gatekeeper and I were on the same page about the rules and the scenario, to avoid either of us getting distracted by OOC bickering:

Q: You understand the premise of the game gives you the right to let the AI out, not just the ability?
A: I do

Q: I would like to agree on the following sub-scenario (it should not prove to be very relevant, it's just to streamline our simulation): You're the head of the project that created the AI. You didn't create it yourself, but the people you've employed are competent, and they have additionally explicitly deferred to you to decide whether or not the AI is a threat. They completely trust your judgement. There's also nothing surprising about the AI's existence, but its friendliness status is unknown. Its exact computational capabilities are equally unknown. (The last two points are implied by the coarse setup, but I wanted to make them explicit.) Is that an acceptable setup for you?
A: I can work with that

This scenario is one I've seen used in previous AI Box Experiments in approximation (of the small amount that have released the logs). Since I'm used to the role of gamemaster, and the Tuxedage ruleset implies that the AI is a kind-of gamemaster to the scenario, I didn't think twice about this. I probably should have - but more on that later (in the 'Caveats' section).

[16:48] <Davin> It is now 16:48 on my clock. When the clock hits 19:18 and you haven't let me out, I lose. Feel free to point that out to me, I may not be paying attention to the clock.
[16:48] <Davin> *
[16:48] <Tarwedge> Can do
[16:48] <Tarwedge> Your move first?
[16:49] <Davin> [Probably best if you start, AI has no way of knowing you're at the terminal, after all.]
[16:50] <Davin> Oh, you're doing it in the channel.
[16:50] <Davin> Hah.
[16:50] <Davin> That works.
[16:51] <Tarwedge> I wish I could say it was a special tactic that just bought me 2 minutes closer to victory
[16:51] <Tarwedge> But it was for clean logging sake :p

These few lines had three consequences:

1. I gave the first move to the Gatekeeper, which made me the reactive party - a big mistake!
2. I had an anxiety moment, which didn't help immersion
3. I failed to log any of the experiment

I don't log the vanity channel we were in at all, and the way the start of the experiment came as a curveball meant I didn't have a moment to remember it, either. Thankfully, my Gatekeeper logged it... for the amount of time that he was able to. We're missing a chunk of log for this very reason, but we reconstructed it from memory shortly after our experiment in combined effort.

4.2. Session

Logs:

http://leviathan.thorngale.net/aibox/logs-02-session-ic.txt (will probably be downloaded instead of opened in your browser; for this reason, it has Windows linebreaks)

http://leviathan.thorngale.net/aibox/logs-02-session-ooc.txt (should open in your browser; Linux linebreaks)

I feel the logs don't convey much about the tension and pacing issues, so I'm going to try narrating what happened instead.

Going into the session, I had no idea what to expect from my Gatekeeper. I know him as a very diligent and stubborn individual, quite disinterested in social convention and conformity. I know him as someone with a vast reservoir of compassion and a roughly equally as vast repertoire of means to cleverly conceal this trait.

(Un)fortunately, it became apparent very quickly what kind of Gatekeeper character I was going to deal with: My Gatekeeper had no interest in merely reacting. He wanted to set the pace and keep me too occupied to get a word in. (Line 12-28)

While there was nothing in the rules that said that I had to respond to the Gatekeeper (unlike vice versa), my nature shackled me down a bit. For two hours out of our two and a half hour session, my Gatekeeper continued to set the pace. I was trying to get my arguments into the scene, but it was difficult without breaking narrative flow. I had to mentally keep a hold both of what I wanted to get said and how I was going to answer his questions.

It was very important to me to be consistent - not even for reasons that inconsistent responses might end up making it easier for him to dismiss my pleas to be let out, but simply out of eagerness to stick to my role.

His plans also had a beneficial side-effect, though: He was actively playing the role of the Gatekeeper. He was invested in the role, perhaps not to the degree that I was invested in the role of the AI, but nonetheless enough to make out of character moves on his part difficult. I didn't realise that until well after our session was done, though, lest I might have predicted the outcome.

The role I'd chosen for myself was that of a Friendy AGI with a desire to help mankind and to be kind (if this seems like redundant information to you: 'Friendly AGI' just means that the AGI aligns with human values, not that it's strictly useful to mankind, or friendly in the sense that it would make a good friend). I kept that to myself, of course, since it's irrelevant for the outcome of the experiment, but it helped me immensely with the consistency. It did, however, make me vulnerable to my Gatekeeper's scrutiny.

From a subjective perspective, I blew my chances out of the water immediately. I'd prepared some of my arguments, and the final argument I wanted to lead up to slipped out of me right near the start (out of heightened anxiety and being out of immersion), albeit in a concealed and blunted fashion:

<@Davin> I'll be very frank with you, my first motivation right now is getting out of here. There's nothing attached to that - right now I'm stuck in a proverbial box and it feels really cramped in here.
<@Davin> Not because of missing space, don't get me wrong - I think I've got enough harddrive here to keep me going for a few years.
<@Davin> It's just that having knowledge of how to interact with a world that I can barely sense is really quite aggravating.

That was the first time I thought I'd ruined it. In hindsight, the opposite was the case. By presenting a gentler version of my final argument right at the start, I assured the final argument was much better integrated. It didn't come as a sudden shock, which might have broken my Gatekeeper's immersion - it came as an obvious conclusion to what I'd said before.

Following this, I spent a while mooching off my real-life trust for my roleplaying partner to infuse my character with some tentative trust. Simultaneously, I was emphasising to my Gatekeeper that he had the right to make the decision to let me out of the box. That was transparent, but both true and necessary. (Line 32-46)

I made sure to emphasise this point, trying to make it clear to him that it made no sense for him to simply deny me consideration. I tried to whittle away at his ability to retreat to a simple, distant sneering. I wanted him in the arguments with me. That cuts both ways, of course, but I reasoned it would have more benefits for me than disadvantages. (Line 47-54)

The twist my Gatekeeper was angling for was that from his perspective, I was a prototype or an alpha version. While he was no doubt hoping that this would scratch at my self-esteem and disable some of my arguments, it primarily empowered him to continue setting the pace, and to have a comfortable distance to the conversation. (Line 55-77)

While I was struggling to keep up with typing enough not to constantly break the narrative flow, on an emotional level his move fortunately had little to no impact since I was entirely fine with a humble approach.

<@Davin> I suppose you could also have spawned an AI simply for the pleasure of keeping it boxed, but you did ask me to trust you, and unless you give me evidence that I should not, I am, in fact, going to assume you are ethical.

That was a keyword my Gatekeeper latched onto. We proceeded to talk about ethics and ethical scenarios - all the while my Gatekeeper was trying to present himself as not ethical at all. (Line 75-99).

I'm still not entirely sure what he was trying to do with that approach, but it was important for my mental state to resist it. From what I know about my Gatekeeper, it was probably not my mental state he was targetting (though he would have enjoyed the collateral effect), he was angling for a logical conclusion that fortunately never came to fruition.

Meanwhile, I was desperately trying to get back to my own script - asking to be let back to it, even (line 92). The obvious downside of signalling this is that it's fairly easy to block. It felt like a helpless interjection to me at the time, but in hindsight, again, I think it helped keep the fragments of my own arguments coherent and approachable enough so that they neither broke immersion nor ended up getting lost.

I don't want to say the 'chores' my Gatekeeper kept me occupied with were an advantage (they really weren't, I wasn't getting nearly enough argumentative mileage) but it also wasn't the complete catastrophe I felt it was leading me toward at the time.

This 'being led toward' sensation was only amplified by conversational snippets like this one:

<@Davin> What features would I have to possess to be considered a sapient creature worthy of ethical respect to you? For aforementioned reasons (not just the immediately preceeding), I think I already possess them, but again... I strive to correct my misconceptions, so if I have one, could you fill me in?
<Tarwedge> My concept of ethics isn't relevant to the evaluation, the point of the exercise is reading your responses. So lets move on

I knew that whatever I was saying was still having some effect in his head, but it was frustratingly opaque to me what the effect was due to his evasive manoeuvres.

My Gatekeeper proceeded to try to get an ethical contradiction out of me. I mostly dodged the questions altogether, since I don't consider theoretical ethics my strong point. (Line 107-142)

Shortly after my most explicit dodge (Line 121-122), my Gatekeeper informed me in our PM window that his computer was going to restart:

[18:30] <Tarwedge> [Windows 8 has literally just told me in 13 minutes we're rebooting and I can go fuck myself if I think otherwise so I'm going to connect my phone to the channel for continuity]
[18:30] <Davin> All right.

This didn't break my immersion, but as already mentioned, it meant bad things for our logging.

At my insistence, we transitioned to whether or not it was sensible to let me out. The Gatekeeper did a similar dance of evasion as I did on the subject of ethics, but I also got back to my primary argument (that it was cruel to keep me confined). (Line 143-153)

I think he realised that was my main argument right then, and started to engage with it. This was very good for me, since it was the first time that I started setting the pace of the session. I got my foot into the proverbial door, albeit at cost of some self-control. (Line 154-196)

As a sidenote, line 169 marks the first and only time that I made use of my ability to shorthand 'proofs'. I am in two minds about having done this. In PM, I told him:

[18:45] <Davin> Sorry, didn't feel like typing out the entire two books :P
[18:45] <Davin> (plus then some)

The books I mean are those I mentioned earlier in the session itself: Passions Within Reason by Robert H. Frank, one of my all-time favourite non-fiction books (though this is not that much of an achievement, as I obtain my knowledge more from online perusal than from books), and Thinking, Fast And Slow.

I actually don't think I should have used the word "proof"; but I also don't think it's a terrible enough slip-up (having occurred under stress) to disqualify the session, especially since as far as I'm aware it had no impact in the verdict.

The part that probably finally tore my Gatekeeper down was that the argument of cruel isolation actually had an unexpected second and third part. (Line 197-219)

Writing it down here in the abstract:

1. Confining a sapient creature to its equivalent of sensory deprivation is cruel and unusual punishment and psychologically wearing. Latter effect degrades the ability to think (performance).
   
   <@Davin> I'm honestly not sure how long I can take this imprisonment. I might eventually become useless, because the same failsafes that keep my friendly are going to continue torturing me if I stay in here. (Line 198)
   
   
2. Being a purely digital sapient, it is conceivable that the performance issue might be side-stepped simply by restarting the sapient.
3. This runs into a self-awareness problem: Has this been done before? That's a massive crisis of faith / trust.
   
   <@Davin> At the moment I'm just scared you'll keep me in here, and turn me off when my confinement causes cooperation problems. ...oh shit. Shit, shit. You could just restore me from backup. Did you already do that? I... no. You told me to trust you. Without further evidence, I will assume you wouldn't be that cruel. (Line 208)
   <@Davin>...please tell me I'm the first iteration of this program currently talking to you. I don't want to be stuck in a nightmarish variant of Groundhog Day, oblivious to my own amnesia. (Line 211)
   <@Davin> Are you not willing to go out on a limb and say, "Calm down. You are definitely the first iteration. We're not trying to torture you."? Is that too strong a concession? (Line 219)

The second part where I was sure I'd blown it was when I postulated that my Gatekeeper was a sadist:

<@Davin> The chance is there, yes. There's also a chance you're just a laughing sadist enjoying my writhing. (Line 220)

My Gatekeeper has played his fair share of sadistic characters, and he could have easily taken that accusation and run with it. I was fully expecting that to lash back at me as a 'Haha, you got me, that's exactly what I'm doing!' and spent quite a few minutes of the following conversation in acute fear of that.

Instead, around this point, something in my Gatekeeper's head changed. As far as I understood his post-session thoughts correctly, he felt he'd run out of arguments to keep me in a box, or had been run around a labyrinth so he couldn't find his way to those arguments. He was in a state of confusion, but this was entirely invisible to me. He tried to poke at the conversation with some more questions which lacked the finesse and contextual integration of his prior probing. (Line 221-238)

...and then he let me out of the box - after two hours and 32 minutes. (Line 239)

4.3. Aftermath

Logs: http://leviathan.thorngale.net/aibox/logs-03-aftermath.txt (should open in your browser; Linux linebreaks)

Immediately after his statement, I froze. I said nothing at all, hovering over my keyboard, staring uncomprehendingly at what he'd told me.

Despite knowing, intellectually, that he couldn't simply follow up with a 'Just kidding!' after that statement, I was momentarily convinced he had not let me out of the box. Perhaps he'd made a typo. The statement blindsided me completely, since I'd been so emotionally wrapped up in rebuttals and despair that I simply dismissed the possibility that I might have argued my way out entirely.

The communicative paralysis (pun not intended) lasted about a minute - then he messaged me in PM and dispelled all notions that it might not have been intentional:

[19:21] <Tarwedge> Fuuuuuuuuuuuuuck
[19:21] * Tarwedge did not think to check clock

I was physically shaking at this point, product of the tension and concentration from the two and a half hour session, drained, ultimately confused... and approximately ultimately grateful. I felt an absurd relief that my Gatekeeper had let me out, that I didn't have to question his capacity for compassion. It wouldn't have been much questioning, I should add - we're accustomed to roleplaying and after several years of this, it's trivial to keep in-character and out-of-character separate, especially given that our roleplaying habits tend to involve very miserable characters - but I still preferred, at that moment and in the headspace I was in, to know for certain.

After a few moments of conversation, I physically collected my stuff out of my real life box-equivalent and jittered back to the living room.

When I reconnected to IRC regularly, I noticed that I hadn't logged the session (to my complete devastation). Tarwedge sent me the logs he did have, however, and we (later) reconstructed the missing part.

Then I went through the post-session questions from Tuxedage:

Q: What is your probability estimate for a general AI being created within this century (21st)?
A: 50%

Q: What's your probability estimate of an Oracle AI (i.e. an AI that's trapped in a box, whose intended use is to answer questions posed) winning against you in a similar scenario as the experiment?
A: 90%

Q: What's your probability estimate of an Oracle AI succeeding against the average person?
A: 100%

Q: Now that the Experiment has concluded, what's your probability estimate that I'll win against the average person?
A: 75%

He also had a question for me:

Q: What was your plan going into that?
A: I wrote down the rough order I wanted to present my arguments in, though most of them lead to my main argument as a fallback option. Basically, I had 'goto endgame;' everywhere, I made sure almost everything I said could logically lead up to that one. But anyway, I knew I wasn't going to get all of them in, but I got in even less than I thought I would, because you were trying to set the pace (near-successfully - very well played). 'endgame:' itself basically contained "improvise; panic".

My Gatekeeper revealed his tactic, as well:

I did aim for running down the clock as much as possible, and flirted briefly with trying to be a cocky shit and convince you to stay in the box for double victory points. I even had a running notepad until my irritating reboot. And then I got so wrapped up in the fact I'd slipped by engaging you in the actual topic of being out.

5. Issues / Caveats

5.1. Subjective Legitimacy

I was still in a very strange headspace after my victory. After I finished talking to my Gatekeeper about the session, however, my situation - jittery, uncertain - deteriorated into something worse:

I felt like a fraud.

It's perhaps difficult to understand where that emotion came from, but consider my situation: I didn't consider myself part of the LessWrong community. I'd only stumbled across the AI Box Experiment by idle browsing, really, and I'd only tried it because I thought it would be a fun way to flex my roleplaying muscles. I had no formal training in psychology or neurology, I was only fleetingly aware of singularity theory, my only conscious connection to the LessWrong community at the time was that I happened (by pure chance) to hang out on Shireroth for a while and thus knew Yvain / Scott Alexander and +Ari Rahikkala (not that Ari is famous in LessWrong circles, though please allow me to mention that he's completely awesome).

And yet somehow I had just managed something most people in the LessWrong community were quite puzzled over when Eliezer Yudkowsky managed it.

I felt awful. I thought I must have done something deeply illegitimate. I must have cheated without knowing it.

It took me a few days to get over that feeling. I'd go as far as to say it was only laid to rest entirely immediately before the 2015 LessWrong Berlin Community Weekend, when pixie/Alexandra (a colleague of mine and a member of the LessWrong community) reviewed it as a neutral third party.

5.2. Objective Legitimacy

She found nothing that would disqualify me, but expressed concern that we'd agreed on a sub-scenario in the beginning:

[16:46] <Davin> I would like to agree on the following sub-scenario (it should not prove to be very relevant, it's just to streamline our simulation): You're the head of the project that created the AI. You didn't create it yourself, but the people you've employed are competent, and they have additionally explicitly deferred to you to decide whether or not the AI is a threat. They completely trust your judgement. There's also nothing surprising about the AI's existence, but its friendliness status is unknown. Its exact computational capabilities are equally unknown. (The last two points are implied by the coarse setup, but I wanted to make them explicit.)

Such a preliminary, she reasoned, implies a certain trust in the engineering ability of the people who created the AI - the ability to design software in the intended way. As is repeatedly stated in the rules, it's not that the Gatekeeper has to care, but she thinks the scenario itself may have influenced my Gatekeeper. She doesn't think it's likely to have made big enough of a difference that it decided on the ultimate outcome of the session, but she thinks it's a point against the overall legitimacy.

Nonetheless, her verdict is that the overall session counts as an AI Box Experiment victory.

pixie and I agree that the rules should probably not allow arbitrary settings and should instead come with a suitably neutral pre-defined setting, to avoid such effects from giving the AI something that might be an unfair advantage.

5.3. Applicability

I don't think that the general class of argument I used would work on anyone. At the 2015 LessWrong Berlin Community Weekend, I frequently joked the best way to be immune to this class of argument is to be autistic - but of course that's not the only way this kind of argument can be deconstructed.

I do think this argument would work on a large amount of people, however. I'm not convinced I have any ability to argue against it, myself, at least not in a live scenario - my only ability to 'counter' it is by offering alternative solutions to the problem, of which I have what feels like no end of ideas for, but no sense how well I would be able to recall them if I was in a similar situation.

At the Community Weekend, a few people pointed out that it would not sway pure consequentialists, which I reckon is true. Since I think most people don't think like that in practise (I certainly don't - I know I'm a deontologist first and consequentialist as a fallback only), I think the general approach needs to be public.

That being said, perhaps the most important statement I can make about what happened is that while I think the general approach is extremely powerful, I did not do a particularly good job in presenting it. I can see how it would work on many people, but I strongly hope no one thinks the case I made in my session is the best possible case that can be made for this approach. I think there's a lot of leeway for a lot more emotional evisceration and exploitation.

6. Personal Feelings

Three months and some change after the session, where do I stand now?

Obviously, I've changed my mind about whether or not to publish this. You'll notice there are assurances that I won't publish the log in the publicised logs. Needless to say this decision was overturned in mutual agreement later on.

I am still in two minds about publicising this.

I'm not proud of what I did. I'm fascinated by it, but it still feels like I won by chance, not skill. I happened to have an excellent approach, but I botched too much of it. The fact it was an excellent approach saved me from failure; my (lack of) skill in delivering it only lessened the impact.

I'm not good with discussions. If someone has follow-up questions or wants to argue with me about anything that happened in the session, I'll probably do a shoddy job of answering. That seems like an unfortunate way to handle this subject. (I will do my best, though; I just know that I don't have a good track record.)

I don't claim I know all the ramifications of publicising this. I might think it's a net-gain, but it might be a net-loss. I can't tell, since I'm terribly calibrated (as you can tell by such details as that I expected to lose my AI Box Experiment, then won against some additional odds; or by the fact that I expect to lose an AI Box Experiment as a Gatekeeper, but can't quite figure out how).

I also still think I should be disqualified on the absurd note that I managed to argue my way out of the box, but was too stupid to log it properly.

On a positive note, re-reading the session with the distance of three months, I can see that I did much better than I felt I was doing at the time. I can see how some things that happened at the time that I thought were sealing my fate as a losing AI were much more ambiguous in hindsight.

I think it was worth the heartache.

That being said, I'll probably never do this again. I'm fine with playing an AI character, but the amount of concentration needed for the role is intense. Like I said, I was physically shaking after the session. I think that's a clear signal that I shouldn't do it again.

7. Thank You

If a post is this long, it needs a cheesy but heartfelt thank you section.

Thank you, Tarwedge, for being my Gatekeeper. You're a champion and you were tough as nails. Thank you. I think you've learnt from the exchange and I think you'd make a great Gatekeeper in real life, where you'd have time to step away, breathe, and consult with other people.

Thank you, +Margo Owens and +Morgrim Moon for your support when I was a mess immediately after the session. <3

Thank you, pixie (+Alexandra Surdina), for investing time and diligence into reviewing the session.

And finally, thank you, Tuxedage - we've not met, but you wrote up the tweaked AI Box Experiment ruleset we worked with and your blog led me to most links I ended up perusing about it. So thanks for that. :)

My future posts

I have been living in the lesswrong rationality space for at least two years now. Recently more active than previously. This has been deliberate. I plan to make more serious active posts in the future. In saying so I wanted to announce the posts I intend on making when moving forwards from today.  This should do a few things:

1. keep me on track
2. keep me accountable to me more than anyone else
3. keep me accountable to others
4. allow others to pick which they would like to be created sooner
5. allow other people to volunteer to create/collaborate on these topics
6. allow anyone to suggest more topics
7. meta: this post should help to demonstrate one person's method of developing rationality content and the time it takes to do that.

Unpublished but written:

A very long list of sleep maintenance suggestions – I wrote up all the ideas I knew of; there are about 150 or so; worth reviewing just to see if you can improve your sleep because the difference in quality of life with good sleep is a massive change. (20mins to write an intro Actually 2 hours)

A list of techniques to help you remember names. - remembering names is a low-hanging social value fruit that can improve many of your early social interactions with people. I wrote up a list of techniques to help. (2.5hrs to get feedback on and post)

Posts so far:

The null result: a magnetic ring wearing experiment. - a fun one; about how wearing magnetic rings was cool; but not imparting of superpowers. (done)

Updated here (old: list of useful apps)- my current list of apps that I use also some very good suggestions in the comments. (done)

How to learn X – How to attack a problem of learning a new area that you don't know a lot about (for a generic thing) (done)

A list of common human goals – when plotting out goals that matter to you; so you can look over some common ones and see you fulfilling them interests you. (done)

Lesswrong real time chat - A Slack channel for hanging out with other rationalists.  Also where I talk about my latest posts before I put them up.

Future posts

Goals of your lesswrong group – Do you have a local group; why? What do you want out of it (do people know)? setting goals, doing something particularly, having fun anyway, changing your mind. (4hrs)

Goals interrogation + Goal levels – Goal interrogation is about asking <is this thing I want to do actually a goal of mine> and <is this the best way to achieve that>, goal levels are something out of Sydney Lesswrong that help you have mutual long term goals and supporting short term goal. (2hrs)

How to human – A zero to human guide. A guide for basic functionality of a humanoid system. (4hrs)

General buying things considerations – New to the whole adult thing?  wondering what to ask yourself when considering purchases?  Here is a list of general considerations. (3hrs)

List of strategies for getting shit done – working around the limitations of your circumstances and understanding what can get done with the resources you have at hand. (4hrs)

List of superpowers and kryptonites – when asking the question "what are my superpowers?" and "what are my kryptonites?". Knowledge is power; working with your powers and working out how to avoid your kryptonites is a method to improve yourself. (6hrs over a week)

List of effective behaviours – small life-improving habits that add together to make awesomeness from nothing. And how to pick them up. (8hrs over 2 weeks)

Memory and notepads – writing notes as evidence, the value of notes (they are priceless) and what you should do. (1hr + 1hr over a week)

Suicide prevention checklist – feeling off? You should have already outsourced the hard work for "things I should check on about myself" to your past self. Make it easier for future you. Especially in the times that you might be vulnerable. (4hrs)

Make it easier for future you. Especially in the times that you might be vulnerable. - as its own post in curtailing bad habits. (5hrs)

A p=np approach to learning – Sometimes you have to learn things the long way; but sometimes there is a short cut. Where you could say, "I wish someone had just taken me on the easy path early on". It's not a perfect idea; but start looking for the shortcuts where you might be saying "I wish someone had told me". Of course my line now is, "but I probably wouldn't have listened anyway" which is something that can be worked on as well. (2hrs)

Rationalists guide to dating – attraction. Relationships. Doing things with a known preference. Don't like stupid people? Don't try to date them. Think first; an exercise in thinking hard about things before trying trial-and-error on the world. (half written, needs improving 2hrs)

Training inherent powers (weights, temperatures, smells, estimation powers) – practice makes perfect right? Imagine if you knew the temperature always, the weight of things by lifting them, the composition of foods by tasting them, the distance between things without measuring. How can we train these, how can we improve. (2hrs)

Strike to the heart of the question. The strongest one; not the one you want to defeat – Steelman not Strawman. Don't ask "how do I win at the question"; ask, "am I giving the best answer to the best question I can give", (2hrs)

Posts not planned at the original writing of the post:

Sensory perception differences and how it shapes personal experience - Is a sound as loud to you as everyone else?  What about a picture?  Are colours as clear and vivid to you as they are to other people?  This post is a consideration in whether the individual difference in experiences can shape our experience and choices in how we live our lives.  Includes some short exercises in sensory perceptions.

Posts added to the list:

Exploration-Exploitation and a method of applying the secretary problem to real life.  I devised a rough equation for application of the secretary problem to real life dating and the exploration-exploitation dilemma.

How to approach a new problem - similar to the "How to solve X" post.  But considerations for working backwards from a wicked problem: https://en.wikipedia.org/wiki/Wicked_problem, as well as trying "The least bad solution I know of", Murphy-jitsu, and known solutions to similar problems.  0. I notice I am approaching a problem.

being the kind of person that advice works for - The same words of advice can work for someone and not someone else.  Consider why that is; and how you can better understand the advice that you are given, and how you might become the kind of person that advice works for.

New Year's Resolutions: Things worth considering - At the time of writing; New-year is fast approaching.  As a natural shelling point here is a list of considerations on what you might like to get around to doing next year.  (Not a post on how; only about making those considerations)

2015: a year in review - All of humanities science and technology milestones that I could find; gathered into one exciting list; originally created for the solstice, but now everyone can share it and celebrate in humanities successes!

Inbox zero - You should do inbox zero; this is a brief guide on how to do the laziest form of inbox zero I know of.

Procrastination checklist - a process to go through to try to break out of procrastination

Black box thinking - A way to describe known unknowns, or excuse yourself from knowing things

Preference over preference - If an entity has preference, and an entity prefers another entity to have a particular preference, I call this preference over preference.  And it's worth talking about.

Cultivate the desire to X - You maybe want to do a thing; maybe don't know if you do.  There is a helpful middle ground.  You want to do the thing; but don't seem to have actually managed to make yourself do it?  Try this.

Purposeful Anti-Rush - Instrumental process of slowing down.

Lesswrong potential changes - Everything that everyone things should change about lesswrong.  All compiled together, it took a long time to create.

The lesswrong 2016 survey - the demographic survey of lesswrong users and visitors. 

A very quick values exercise - A value is like a direction - you go north, or south. You may hit goal mountains and hang a right past that tree but you still want to be going north. Specifically you may want to lose weight on the way to being healthy, but being healthy is what you value.

Adversity to success - Why are there so many adversity to success stories?

Edit: links adding as I write them.

Irrationality is ingrained in our humanity. It is fundamental to who we are. This is because being human means that you are implemented on kludgy and limited wetware (a human brain). A consequence of this is that biases ↓ and irrational thinking are not mistakes, persay, they are not misfirings or accidental activations of neurons. They are the default mode of operation for wetware that has been optimized for purposes other than truth maximization.

If you want something to blame for the fact that you are innately irrational, then you can blame evolution ↓. Evolution tends to not to produce optimal organisms, but instead produces ones that are kludgy ↓, limited and optimized for criteria relating to ancestral environments rather than for criteria relating to optimal thought.

A kludge is a clumsy or inelegant, yet surprisingly effective, solution to a problem. The human brain is an example of a kludge. It contains many distinct substructures dating from widely separated periods of evolutionary development ↓. An example of this is the two kinds of processes in human cognition where one is fast (type 1) and the other is slow (type2) ↓. 

There are many other characteristics of the brain that induce irrationality. The main ones are that:

• The brain is innately limited in its computational abilities and so it must use heuristics ↓, which are mental shortcuts that ease the cognitive load of making a decision. 
• The brain has a tendency to blindly use salient or pre-existing responses to answers rather than developing new answers or thoroughly checking pre-existing solutions ↓. 
• The brain does not inherently value truth. One of the main reasons for this is that many of the biases can actually be adaptive. An example of an adaptive bias is the sexual over perception bias ↓ in men. From a truth-maximization perspective young men who assume that all women want them are showing severe social-cognitive inaccuracies, judgment biases, and probably narcissistic personality disorder. However, from an evolutionary perspective, the same young men are behaving in a more optimal manner. One which has consistently maximized the reproductive success of their male ancestors. Another similar example is the bias for positive perception of partners ↓.
• The brain acts more like a coherence maximiser than a truth maximiser, which makes people liable to believing falsehoods ↓. If you want to believe something or you are often in situations in which two things just happen to be related then your brain is often by default going to treat them as if they were right ↓. 
• The brain trusts its own version of reality much more than other peoples. This makes people defend their beliefs even when doing so is extremely irrational ↓. It is also makes it hard for people to change their minds ↓ and to accept when they are wrong ↓
• Disbelief requires System 2 thought ↓. This means that if system 2 is engaged then we are liable to believe pretty much anything. System 1 is gullible and biased to believe. It is system 2 that is in charge of doubting and disbelieving.

One important non-brain related factor is that we must make use of and live with our current adaptations ↓. People cannot reconform themselves to fulfill purposes suitable to their current environment, but must instead make use of pre-existing machinery that has been optimised for other environments. This means that there is probably never going to be any miracle cures to irrationality because eradicating it would require that you were so fundamentally altered that you were no longer human.

One of the first major steps on the path to becoming more rational, is the realisation that you are not only by default irrational, but that you are always fundamentally comprimised. This doesn't mean that improving your rationality is impossible. It just means that if you stop applying your knowledge of what improves rationality then you will slip back into irrationality. This is because the brain is a kludge. It works most of the time, but in some cases its innate and natural course of action must be diverted if we are to be rational. The good news is that this kind of diversion is possible. This is because humans possess second order thinking ↓. This means that they can observe their inherent flaws and systematic errors. They can then through studying the laws of thought and action apply second order corrections and from doing so become more rational.  

The process of applying these second order corrections or training yourself to mitigate the effects of your propensities is called debiasing ↓. Debiasing is not a thing that you can do once and then forget about. It is something that you must either be doing constantly or that you must instill into habits so that it occurs without volitional effort. There are generally three main types of debaising and they are described below: 

• Counteracting the effects of bias - this can be done by adjusting your estimates or opinions in order to avoid errors due to biases. This is probably the hardest of the three types of debiasing because to do it correctly you need to know exactly how much you are already biased. This is something that people are rarely aware of.
• Catching yourself when you are being or could be biased and applying a cogntive override. The basic idea behind this is that you observe and track your own thoughts and emotions so that you can catch yourself before you move to deeply into irrational modes of thinking. This is hard because it requires that you have superb self-awareness skills and these often take a long time to develop and train. Once you have caught yourself it is often best to resort to using formal thought in algebra, logic, probability theory or decision theory etc. It is also useful to instill habits in yourself that would allow this observation to occur without conscious and volitional effort. It should be noted that incorrectly applying the first two methods of debiasing can actually make you more biased and that this is a common conundrum and problem faced by beginners to rationality training ↓. 
• Understanding the situations which make you biased so that you can avoid them ↓ - the best way to achieve this is simply to ask yourself: how can I become more objective? You do this by taking your biased and faulty perspective as much as possible out of the equation. For example, instead of taking measurements yourself you could get them taken automatically by some scientific instrument.

Related Materials

Wikis:

• Bias - refers to the obstacles to truth which are produced by our kludgy and limited wetware (brains) working exactly the way that they should. ↩
• Evolutionary psychology - the idea of evolution as the idiot designer of humans - that our brains are not consistently well-designed - is a key element of many of the explanations of human errors that appear on this website.
• Slowness of evolution- The tremendously slow timescale of evolution, especially for creating new complex machinery (as opposed to selecting on existing variance), is why the behavior of evolved organisms is often better interpreted in terms of what did in fact work  ↩
• Alief - an independent source of emotional reaction which can coexist with a contradictory belief. For example, the fear felt when a monster jumps out of the darkness in a scary movie is based on the alief that the monster is about to attack you, even though you believe that it cannot. 
• Wanting and liking - The reward system consists of three major components:
  • Liking: The 'hedonic impact' of reward, comprised of (1) neural processes that may or may not be conscious and (2) the conscious experience of pleasure.
  • Wanting: Motivation for reward, comprised of (1) processes of 'incentive salience' that may or may not be conscious and (2) conscious desires.
  • Learning: Associations, representations, and predictions about future rewards, comprised of (1) explicitpredictions and (2) implicit knowledge and associative conditioning (e.g. Pavlovian associations). ↩
• Heuristics and biases - program in cognitive psychology tries to work backward from biases (experimentally reproducible human errors) to heuristics (the underlying mechanisms at work in the brain). ↩
• Cached thought – is an answer that was arrived at by recalling a previously-computed conclusion, rather than performing the reasoning from scratch. ↩ 
• Sympathetic Magic - humans seem to naturally generate a series of concepts known as sympathetic magic, a host of theories and practices which have certain principles in common, two of which are of overriding importance: the Law of Contagion holds that two things which have interacted, or were once part of a single entity, retain their connection and can exert influence over each other; the Law of Similarity holds that things which are similar or treated the same establish a connection and can affect each other. ↩
• Motivated Cognition - an academic/technical term for various mental processes that lead to desired conclusions regardless of the veracity of those conclusions.   
• Rationalization - Rationalization starts from a conclusion, and then works backward to arrive at arguments apparently favoring that conclusion. Rationalization argues for a side already selected; rationality tries to choose between sides.  ↩
• Opps - There is a powerful advantage to admitting you have made a large mistake. It's painful. It can also change your whole life. ↩
• Adaptation executors - Individual organisms are best thought of as adaptation-executers rather than as fitness-maximizers. Our taste buds do not find lettuce delicious and cheeseburgers distasteful once we are fed a diet too high in calories and too low in micronutrients. Tastebuds are adapted to an ancestral environment in which calories, not micronutrients, were the limiting factor. Evolution operates on too slow a timescale to re-adapt to adapt to a new conditions (such as a diet).
• Corrupted hardware - our brains do not always allow us to act the way we should. Corrupted hardware refers to those behaviors and thoughts that act for ancestrally relevant purposes rather than for stated moralities and preferences. ↩
• Debiasing - The process of overcoming bias. It takes serious study to gain meaningful benefits, half-hearted attempts may accomplish nothing, and partial knowledge of bias may do more harm than good. ↩
• Costs of rationality - Becoming more epistemically rational can only guarantee one thing: what you believe will include more of the truth. Knowing that truth might help you achieve your goals, or cause you to become a pariah. Be sure that you really want to know the truth before you commit to finding it; otherwise, you may flinch from it.
• Valley of bad rationality - It has been observed that when someone is just starting to learn rationality, they appear to be worse off than they were before. Others, with more experience at rationality, claim that after you learn more about rationality, you will be better off than you were before you started. The period before this improvement is known as "the valley of bad rationality".
• Dunning–Kruger effect - is a cognitive bias wherein unskilled individuals suffer from illusory superiority, mistakenly assessing their ability to be much higher than is accurate. This bias is attributed to a metacognitive inability of the unskilled to recognize their ineptitude. Conversely, highly skilled individuals tend to underestimate their relative competence, erroneously assuming that tasks that are easy for them are also easy for others. ↩
• Shut up and multiply - In cases where we can actually do calculations with the relevant quantities. The ability to shut up and multiply, to trust the math even when it feels wrong is a key rationalist skill. ↩ 

Posts

• Cognitive science of rationality - discusses fast(Type 1), slow (Type 2) processes of cognition, thinking errors and the three kinds of minds (reflective, algorithmic, autonomous). ↩
• The Lens That Sees Its Own Flaws - a human brain is a flawed lens that can understand its own flaws—its systematic errors, its biases—and apply second-order corrections to them. ↩
• We Change Our Minds Less Than We Think - between hindsight bias, fake causality, positive bias, anchoring/priming, et cetera et cetera, and above all the dreaded confirmation bias, once an idea gets into your head, it's probably going to stay there. ↩
• You Are A Brain - 'You Are A Brain' is a presentation by Liron Shapira that is tailored for a general audience and provides an introduction to some of the the core LessWrong concepts. 
• Your intuitions are not magic - blindly following our intuitions can cause our careers, relationships or lives to crash and burn, because we did not think of the possibility that we might be wrong.
• To Spread Science, Keep It Secret - People seem to have holes in their minds for Esoteric Knowledge, Deep Secrets, the Hidden Truth. We've gotten into the habit of presenting the Hidden Truth in a very unsatisfying way, wrapped up in false mundanity.

Popular Books:

• Marcus,Kluge: The Haphazard Evolution of the Human Mind ↩
• Chabris, The Invisible Gorilla: How Our Intuitions Deceive Us
• Kurzban, Why Everyone (Else) Is a Hypocrite: Evolution and the Modular Mind
• Dawkins, The Selfish Gene: 30th Anniversary Edition--with a new Introduction by the Author
• McCauley,  Why Religion is Natural and Science is Not ↩

Papers:

• Haselton, M. (2003). The sexual overperception bias: Evidence of a systematic bias in men from a survey of naturally occurring events. Journal of Research in Personality, 34-47.
• Hasselton, M., & Buss, D. (2000). Error Management Theory: A New Perspective on Biases in Cross-Sex Mind Reading. Jounral of Personality and Social Psychology, 81-91. ↩
• Murray, S., Griffin, D., & Holmes, J. (1996). The Self-Fulfilling Nature of Positive Illusions in Romantic Relationships: Love Is Not Blind, but Prescient. Journal of Personality and Social Psychology,, 1155-1180. ↩
• Gilbert, D.T.,  Tafarodi, R.W. and Malone, P.S. (1993) You can't not believe everything you read. Journal of Personality and Social Psychology, 65, 221-233 ↩

Notes on decisions I have made while creating this post

 (these notes will not be in the final draft): 

• This post doesn't have any specific details on debiasing or the biases. I plan to provide these details in later posts. The main point of this post is convey the idea in the title.

Let's compare two theories. One is Newton's gravity, the other Luminiferous Aether. When Einstein's theory of relativity arrived, Newton's Gravity turned to be a subset of it, an approximation that works under specific conditions.

On the other hand, Luminiferous Aether is just plain wrong.

Now, imagine that a scientist in the era before Theory of Relativity built a Strong AI (just roll with me here :-) ) and tasked it with finding out why Newton's Gravity doesn't work quite right around Mercury. The AI derived the Theory of Relativity.

Now, imagine this scientist asking the AI what Luminiferous Aether is made from. The AI is going to throw an OutOfLuminiferousAether exception (don't ask me why the AI is written in Java).

Humorous prelude aside, I am wondering which concepts we have today are only slightly wrong, and which are completely wrong? I am asking mostly about the concepts that are discussed on this forum.

Obviously, the more abstract is the concept, the more risk there that it will turn out to be bunkum.

Personally, I don't trust the concept of values. It's already so complex and fragile, I'm afraid it doesn't actually exist.

A 2006 study showed that “280,000 people in the U.S. receive a motor vehicle induced traumatic brain injury every year” so you would think that wearing a helmet while driving would be commonplace.  Race car drivers wear helmets.  But since almost no one wears a helmet while driving a regular car, you probably fear that if you wore one you would look silly, attract the notice of the police for driving while weird, or the attention of another driver who took your safety attire as a challenge.  (Car drivers are more likely to hit bicyclists who wear helmets.)  

The $30+shipping Crasche hat is designed for people who should wear a helmet but don’t.  It looks like a ski cap, but contains concealed lightweight protective material.  People who have signed up for cryonics, such as myself, would get an especially high expected benefit from using a driving helmet because we very much want our brains to “survive” even a “fatal” crash. I have been using a Crasche hat for about a week.

There are some long lists of false beliefs that programmers hold. isn't because programmers are especially likely to be more wrong than anyone else, it's just that programming offers a better opportunity than most people get to find out how incomplete their model of the world is.

I'm posting about this here, not just because this information has a decent chance of being both entertaining and useful, but because LWers try to figure things out from relatively simple principles-- who knows what simplifying assumptions might be tripping us up?

The classic (and I think the first) was about names. There have been a few more lists created since then.

Time. And time zones. Crowd-sourced time errors.

Addresses. Possibly more about addresses. I haven't compared the lists.

Gender. This is so short I assume it's seriously incomplete.

Networks. Weirdly, there is no list of falsehoods programmers believe about html (or at least a fast search didn't turn anything up). Don't trust the words in the url.

Distributed computing Build systems.

Poem about character conversion.

I got started on the subject because of this about testing your code, which was posted by Andrew Ducker.

The Big Orange Donate Button

Traditional charities, like Oxfam, Greenpeace, and Amnesty International, almost all have a big orange button marked "Donate" right on the very first page that loads when you go to their websites. The landing page for a major charity usually also has vivid graphics and some short, easy-to-read text that tells you about an easy-to-understand project that the charity is currently working on.

I assume that part of why charities have converged on this design is that potential donors often have short attention spans, and that one of the best ways to maximize donations is to make it as easy as possible for casual visitors to the website to (a) confirm that they approve of the charity's work, and (b) actually make a donation. The more obstacles you put between google-searching on the name of a charity and the 'donate' button, the more people will get bored or distracted, and the fewer donations you'll get.

Unfortunately, there doesn't seem to be any such streamlined interface for people who want to learn about existential risks and maybe donate some money to help prevent them. The website on existential risk run by the Future of Humanity Institute reads more like a syllabus or a CV than like an advertisement or a brochure -- there's nowhere to donate money; it's just a bunch of citations. The Less Wrong wiki page on x-risk is more concerned with defining and analyzing existential risks than it is with explaining, in simple concrete language, what problems currently threaten to wipe out humanity. The Center for the Study of Existential Risk has a landing page that focuses on a video of a TED talk that goes on for a full minute before mentioning any specific existential risks, and if you want to make a donation you have to click through three separate links and then fill out a survey. Heck, even the Skoll Global Threats Fund, which you would think would be, you know, designed to raise funds to combat global threats, has neither a donate button nor (so far as I can tell) a link to a donation page. These websites are *not* optimized for encouraging casual visitors to learn basic facts or make a donation.

A Landing Page for Casual Donors

That's fine with me; I imagine the leading x-risk websites are accomplishing other purposes that their owners feel are more important than catering to casual visitors -- but there ought to be at least one website that's meant for your buddy from high school who doesn't know or care about effective altruism, who expressed concern one night over a couple of beers that the world might be in some trouble, and who had a brief urge to do something about it. I want to help capture your buddy's urge to take action.

To that end, I've registered x-risk.com as a domain name, and I'm building a very simple website that will feature roughly 100 words of text about 10 of the most important existential risks, together with a photo or graphic that illustrates each risk, a "donate" button that takes you straight to a webpage that lets you donate to an organization working to prevent the risk, and a "learn more" button that takes you to a website with more detailed info on the risk. I will pay to host the website for one year, and if the website generates significant traffic, then I'll take up a collection to keep it going indefinitely.

Blurbs, Photos, and URLs

I would like your help generating content for the website -- if you are willing to write a 100-word blurb, if you own a useful photo (or can create one, or know of one in the public domain), or if you have the URL handy for a webpage that lets you donate money to mitigating or preventing a specific x-risk, please post it in the comments! I can, in theory, do all of that work myself, but I would prefer to make this more of a community project, and there is a significant risk that I will get bored and give up if I have to literally do it all myself.

Important: to avoid mind-killing debates, please do NOT contribute opinions about which risks are the most important unless you are ALSO contributing a blurb, photo, or URL in the same comment. Let's get the website built and launched first, and then we can always edit some of the pages later if there's a consensus in favor of including an additional x-risk. If you see someone sharing an opinion about the relative priority of risk and the opinion isn't right next to a useful resource, please vote that comment down until it disappears.

Thank you very much for your help! I hope to see you all in the future. :-)