There's a semi-famous paper by Dunn, Gilbert and Wilson: "If money doesn't make you happy, then you probably aren't spending it right". (Proper reference: Dunn, E.W., Gilbert, D.T., and Wilson, T.D., If money doesn't make you happy, then you probably aren't spending it right, Journal of Consumer Psychology, vol 21, issue 2, April 2011, pp. 115–125.) It's been referenced a few times on LW but curiously never written up properly here. The purpose of this post is to remedy that.

There is an earlier LW post called "Be Happier" which among other things references this paper and quotes some things it says, but that post is monstrously long and covers a lot more ground (hence, less details on the material in this paper).

Dunn, Gilbert and Wilson (hereafter "DGW") offer eight principles to follow. Here they are.

1. Buy experiences instead of things.

Many studies have asked people to reflect on past "material" and/or "experiential" purchases and have consistently found that they report greater happiness from (and are made happier by recalling) the latter than the former.

Why? DGW propose 5 reasons. First, deliberately sought-out experiences encourage us to focus on the here and now (something shown to increase happiness substantially); second, when things don't change we adapt to them rapidly, and "material" purchases like cars and tables tend to be pretty stable (whereas ongoing experiences are more varied); third, it turns out that people spend more time anticipating experiences before they happen and recalling them afterwards than they do for material purchases. Fourth, experiences are less directly comparable to alternatives than material things, and therefore less subject to post-purchase regret. Fifth, experiences are often shared, and other people are a great source of happiness.

2. Help others instead of yourself.

Prosocial spending correlates better to happiness than personal spending. If you give random people money and either tell them to spend it on themselves or to spend it on someone else, the latter makes them happier. Reflecting on past spending-on-others makes people happier than reflecting on past spending-on-self. (I am a little skeptical about that one: the right point of comparison would be not the past spending but the past enjoyment of whatever you spent the money on.)

Why? DGW propose two reasons. First, prosocial spending is good for relationships and relationships are good for happiness. Second, when you spend on someone else you get to feel like a good person.

Most people have wrong intuitions about this: they expect spending on themselves to make them happier. Most people are wrong.

3. Buy many small pleasures instead of few big ones.

As we saw above under #1, we quickly adapt to changes. Therefore, a larger number of varied small pleasures may be a better buy than a single big one. There is some evidence for this (though to my mind it seems to bear less directly on DGW's principle than in the other cases we've considered so far). If you correlate people's happiness with their positive experiences, the correlation with how frequent those experiences are is stronger than the correlation with how intense they are. The optimal (for happiness) number of sexual partners to have over a year is one, perhaps because that gets you more sex even if individual instances are less exciting. (I find this less than convincing; individual instances might be better because partners learn what works well for them.)

The other reason DGW suggest why more smaller things should be better is diminishing marginal utility: half a cookie is more than half as good as a whole cookie. (This is, I think, partly because of adaptation, but that isn't the whole story.)

DGW suggest that this is one reason why the relationship between wealth and happiness isn't stronger: "wealth promises access to peak experiences, which in turn undermine the ability to savor small pleasures".

4. Buy less insurance.

We adapt to bad things as well as good, which means that bad things are less bad than we are liable to expect. Our overestimation of the impact of adverse occurrences is one reason why we buy insurance, which notoriously is always negative-expectation in monetary terms.

DGW cite various studies showing that people expect to be made markedly unhappier by losses than they actually are if the losses occur, and that people expect to regret bad outcomes more than they actually do (we overestimate how much we will blame ourselves, because we underestimate how good we are at blaming anything and anyone else for our misfortunes).

5. Pay now and consume later.

The opposite of the bargain proposed by credit cards! Besides the purely financial problems that arise from overspending (which are large and widespread), DGW suggest that "consume now, pay later" is bad for our happiness because it eliminates anticipation. We may derive a lot of pleasure even from anticipating something that we don't enjoy very much when it happens. "People who devote time to anticipating enjoyable experiences report being happier in general."

You might think that moving an experience later would simply mean more anticipation (good) but less reminiscence (bad), but it turns out that anticipation generally brings more happiness. (And, for unpleasant events, more pain.)

DGW suggest two other benefits of delaying consumption. First, we may make better choices (meaning, in this case, ones yielding more happiness overall, even if less in the very short term) when we make them a little way ahead. Second, the delay may increase uncertainty, which may keep attention focused on the thing we're buying, which may reduce adaptation. (This seems a little convoluted to me; DGW cite some research backing it up but I'm not sure it backs up the "by reducing adaptation" part of it.)

6. Think about what you're not thinking about.

That is: when choosing what to spend on, take some time to consider less obvious aspects that you'd otherwise be tempted to neglect. "The bigger home may seem like a better deal, but if the fixer-upper requires trading Saturday afternoons with friends for Saturday afternoons with plumbers, it may not be such a good deal after all." And: "consumers who expect a single purchase to have a lasting impact on their happiness might make more realistic predictions if they simply thought about a typical day in their life." (Rather than considering only the small bits of that day that will be impacted by their purchase.)

7. Beware of comparison shopping.

Comparison shopping, say DGW, focuses attention on the features that most clearly distinguish candidate purchases from one another, whereas other more-common features may actually have much more impact on happiness. It may also focus attention on more-concrete differences; for instance, if you ask people whether they would more enjoy a small heart-shaped chocolate or a large cockroach-shaped one, they generally prefer the former, but if you ask them to choose one of the two they tend to focus on the size and choose the latter.

DGW also point out that the context during comparison-shopping tends to be different from that during actual consumption, which can skew our evaluations.

8. Follow the herd instead of your head.

DGW cite research supporting de la Rochefoucauld's advice: "Before we set our hearts too much upon anything, let us first examine how happy those are who already possess it." Others' actual experiences of a thing are likely to be better predictors of our enjoyment than our theoretical estimates: we may know ourselves better, but they know the thing better.

They also suggest (and I don't think this really fits their heading) looking to others for advice on how we would enjoy something we are considering buying. The example they give is of research in which subjects were shown some foods and asked to estimate how much they would enjoy them, after which they ate them and evaluated their actual enjoyment. The wrinkle is that they were also observed, at the moment of being shown the foods, by other observers, who rated their immediate facial reactions -- which turned out to be better predictors of their enjoyment than the subjects' own assessments. So "other people may provide a useful source of information about the products that will bring us joy because they can see the nonverbal reactions that may escape our own notice".

This post won't directly address the Sleeping Beauty problem so you may want to read the above link to understand what the sleeping beauty problem is first.

Half*-Sleeping Beauty Problem

The asterisk is because it is only very similar to half of the sleeping beauty problem, not exactly half.

A coin is flipped. If it is heads, you are woken up with 50% chance and interrogated about the probability of the coin having come up heads. The other 50% of the time you are killed. If it is tails you are woken up and similarly interrogated. Given that you are being interrogated, what is the probability that the coin came up heads? And have you received any new information?

Double-Half*-Sleeping Beauty problem

A coin is flipped. If it is heads, a coin is flipped again. If this second coin is heads you are woken up and interrogated on Monday, if it is tails you are woken up and interrogated on Tuesday. If it is tails, then you are woken up on Monday and Tuesday and interrogated both days (having no memory of your previous interrogation). If you are being interrogated, what is the chance the coin came up heads? And have you received any new information?

Double-Half*-Sleeping Beauty problem with Known Day Variation

Sleeping Couples Problem

A man and his identical-valued wife have lived together for so many years that they have reached Aumann agreement on all of their beliefs, including core premises, so that they always make the same decision in every situation.

A coin is flipped. If it is heads, one of the couple is randomly woken up and interrogated about the probability of the coin having come up heads. The other is killed. If it is tales, both are woken up separately and similarly interrogated. If you are being interrogated, what is the probability that the coin came up heads? And have you received any new information?

Sleeping Clones Problem

A coin is flipped. If it is heads, you are woken up and interrogated about the probability of the coin having come up heads. If it is tails, then you are cloned and both copies are interrogated separately without knowing whether they are the clone or not. If you are being interrogated, what is the probability that the coin came up heads? And have you received any new information?

My expectation is that the Double-Half Sleeping Beauty and Sleeping Clones will be controversial, but I am optimistic that there will be a consensus on the other three.

Solutions (or at least what I believe to be the solutions) will be forthcoming soon.

I've put together a list of what I think are the best Yvain (Scott Alexander) posts for new readers, drawing from SlateStarCodex, LessWrong, raikoth.net, and Scott's LiveJournal.

The list should make the most sense to people who start from the top and read through it in order, though skipping around is encouraged too. Rather than making a chronological list, I’ve tried to order things by a mix of "where do I think most people should start reading?" plus "sorting related posts together."

This is a work in progress; you’re invited to suggest things you’d add, remove, or shuffle around. Since many of the titles are a bit cryptic, I'm adding short descriptions. See my blog for a version without the descriptions.

I. Rationality and Rationalization

• Blue- and Yellow-Tinted Choices  ·····  An introduction to context-sensitive biases.
• The Apologist and the Revolutionary  ·····  Do separate brain processes rationalize and question ideas?
• Historical Realism  ·····  When reality is unrealistic.
• Simultaneously Right and Wrong  ·····  On self-handicapping and self-deception.
• You May Already Be A Sinner  ·····  Self-deception in cases where your decisions make no difference.
• Beware the Man of One Study  ·····  On minimum wage laws and cherry-picked evidence.
• Debunked and Well-Refuted  ·····  When should we say that a study has been "debunked"?
• How to Not Lose an Argument  ·····  How to be more persuasive in entrenched arguments.
• The Least Convenient Possible World  ·····  Why it's useful to strengthen arguments you disagree with.
• Bayes for Schizophrenics: Reasoning in Delusional Disorders  ·····  Hypotheses about the role of perception, evidence integration, and priors in delusions.
• Generalizing from One Example  ·····  On the typical mind fallacy: assuming other people are like you.
• Typical Mind and Politics  ·····  Do political disagreements stem from neurological disagreements?

II. Probabilism

• Confidence Levels Inside and Outside an Argument  ·····  Should you believe your own conclusions, when they're extreme?
• Schizophrenia and Geomagnetic Storms  ·····  When bizarre ideas turn out to be true.
• Talking Snakes: A Cautionary Tale  ·····  Should we dismiss all absurd claims?
• Arguments from My Opponent Believes Something  ·····  Ten fully general arguments.
• Statistical Literacy Among Doctors Now Lower Than Chance  ·····  Common errors in probabilistic reasoning.
• Techniques for Probability Estimates  ·····  Six methods for quantifying uncertainty.
• On First Looking into Chapman’s “Pop Bayesianism”  ·····  Reasons Bayesian epistemology may not be trivial.
• Utilitarianism for Engineers  ·····  Are there good-enough heuristics for comparing people's preferences?
• If It’s Worth Doing, It’s Worth Doing with Made-Up Statistics  ·····  The practical value of probabilities.
• Marijuana: Much More Than You Wanted to Know  ·····  Assessing marijuana's costs and benefits.
• Are You a Solar Deity?  ·····  On confirmation bias in the comparative study of religions.
• The "Spot the Fakes" Test  ·····  An approach to testing humanities hypotheses.
• Epistemic Learned Helplessness  ·····  What should we do when bad arguments sound convincing?

III. Science and Doubt

• Google Correlate Does Not Imply Google Causation  ·····  Peculiar correlations between Google search terms.
• Stop Confounding Yourself! Stop Confounding Yourself!  ·····  A correlational study on the effects of bullying.
• Effects of Vertical Acceleration on Wrongness  ·····  On evidence-based medicine.
• 90% Of All Claims About The Problems With Medical Studies Are Wrong  ·····  Is it the case that "90% of medical research is false"?
• Prisons are Built with Bricks of Law and Brothels with Bricks of Religion, But That Doesn’t Prove a Causal Relationship  ·····  Do psychiatric interventions increase suicide risk?
• Noisy Poll Results and the Reptilian Muslim Climatologists from Mars  ·····  Skepticism about poll results.
• Two Dark Side Statistics Papers  ·····  Statistical tricks for creating effects out of nothing.
• Alcoholics Anonymous: Much More Than You Wanted to Know  ·····  Is AA effective for treating alcohol abuse?
• The Control Group Is Out Of Control  ·····  Parapsychology as the "control group" for all of psychology.
• The Cowpox of Doubt  ·····  Focusing on easy questions inoculates against uncertainty.
• The Skeptic's Trilemma  ·····  Explaining mysteries, vs. worshiping them, vs. dismissing them.
• If You Can't Make Predictions, You're Still in a Crisis  ·····  On psychology studies' replication failures.

IV. Medicine, Therapy, and Human Enhancement

• Scientific Freud  ·····  How does psychoanalysis compare to cognitive behavioral therapy?
• Sleep – Now by Prescription  ·····  On melatonin.
• In Defense of Psych Treatment for Attempted Suicide  ·····   Suicide is usually not a rational, informed decision.
• Who By Very Slow Decay  ·····  On old age and death in the medical system.
• Medicine, As Not Seen on TV  ·····  What is it actually like to be a doctor?
• Searching for One-Sided Tradeoffs  ·····  How can we find good ideas that others haven't found first?
• Do Life Hacks Ever Reach Fixation?  ·····  Why aren't there more good ideas that everyone has adopted?
• Polyamory is Boring  ·····  Deromanticizing multi-partner romance.
• Can You Condition Yourself?  ·····  On shaping new habits by rewarding oneself.
• Wirehead Gods on Lotus Thrones  ·····  Is the future boring? Transcendently blissful? Boringly blissful?
• Don’t Fear the Filter  ·····  Does the Fermi Paradox mean that our species is doomed?
• Transhumanist Fables  ·····  Six futurist fairy tales.

V. Introduction to Game Theory

• Backward Reasoning Over Decision Trees  ·····  Sequential games, and why adding options can hurt you.
• Nash Equilibria and Schelling Points  ·····  Simultaneous games, mixed strategies, and coordination.
• Introduction to Prisoners' Dilemma  ·····  Why Nash equilibria are sometimes bad for everyone.
• Real-World Solutions to Prisoners' Dilemmas  ·····  How society and evolution ensure mutual cooperation.
• Interlude for Behavioral Economics  ·····  Fairness, superrationality, and self-image in real-world games.
• What is Signaling, Really?  ·····  Actions that convey information, sometimes at great cost.
• Bargaining and Auctions  ·····  Idealized models of correct bidding.
• Imperfect Voting Systems  ·····  Strengths and weaknesses of different voting systems.
• Game Theory as a Dark Art  ·····  Ways to exploit seemingly "economically rational" behavior.

VI. Promises and Principles

• Beware Trivial Inconveniences  ·····  Small obstacles can have a huge effect on behavior.
• Time and Effort Discounting  ·····  On inconsistencies in our revealed preferences.
• Applied Picoeconomics  ·····  Binding your future self to your present goals.
• Schelling Fences on Slippery Slopes  ·····  Using arbitrary thresholds to improve coordination.
• Democracy is the Worst Form of Government Except for All the Others Except Possibly Futarchy  ·····  Like democracy, futarchy (rule by prediction markets) has the advantage of appearing impartial.
• Eight Short Studies on Excuses  ·····  When should we allow exceptions to our rules?
• Revenge as Charitable Act  ·····  Revenge can be a personally costly way to disincentivize misdeeds.
• Would Your Real Preferences Please Stand Up?  ·····  Are we hypocrites, or just weak-willed?
• Are Wireheads Happy?  ·····  Distinguishing "wanting" something from "liking" it.
• Guilt: Another Gift Nobody Wants  ·····  An evolutionary, signaling-based explanation of guilt.

VII. Cognition and Association

• Diseased Thinking: Dissolving Questions about Disease  ·····  On verbal disagreements.
• The Noncentral Fallacy — The Worst Argument in the World?  ·····  Judging an entire category by an emotional association that only applies to typical category members.
• The Power of Positivist Thinking  ·····  Focus on statements' empirical content.
• When Truth Isn't Enough  ·····  It's possible to agree denotationally while disagreeing connotationally.
• Ambijectivity  ·····  When a question is both subjective and objective.
• The Blue-Minimizing Robot  ·····  A parable on agency.
• Basics of Animal Reinforcement  ·····  A primer on classical and operant conditioning.
• Wanting vs. Liking Revisited  ·····  Distinguishing motivation to act from reinforcement.
• Physical and Mental Behavior  ·····  Behaviorism meets thinking.
• Trivers on Self-Deception  ·····  The conscious mind as a self-serving social narrative.
• Ego-Syntonic Thoughts and Values  ·····  On endorsed vs. non-endorsed mental behavior.
• Approving Reinforces Low-Effort Behaviors  ·····  Using your self-image to blackmail yourself.
• To What Degree Do We Have Goals?  ·····  Are our unconscious drives like an agent?
• The Limits of Introspection  ·····  Are we good at directly perceiving our cognition?
• Secrets of the Eliminati  ·····  Reducing phenomena to simpler parts, vs. eliminating them.
• Tendencies in Reflective Equilibrium  ·····  Aspiring to become more consistent.
• Hansonian Optimism  ·····  If ego-syntonic goals are about signaling, is goodness a lie?

VIII. Doing Good

• Newtonian Ethics  ·····  Satirizing moral parochialism and sloppy systematizations of ethics.
• Efficient Charity: Do Unto Others...  ·····  How should we act when our decisions matter most?
• The Economics of Art and the Art of Economics  ·····  Should Detroit sell its publicly owned artwork?
• A Modest Proposal  ·····  Using dead babies as a unit of currency.
• The Life Issue  ·····  What are the consequences of drone warfare?
• What if Drone Warfare Had Come First?  ·····  A thought experiment.
• Nefarious Nefazodone and Flashy Rare Side-Effects  ·····  On choosing between drug side-effects.
• The Consequentialism FAQ  ·····  Argues for assessing actions based on how they help or harm people.
• Doing Your Good Deed for the Day  ·····  Doing some good can reduce people's willingness to do more good.
• I Myself Am A Scientismist  ·····  Why apply scientific methods to non-scientific domains?
• Whose Utilitarianism?  ·····  Questioning the objectivity and uniqueness of utilitarianism.
• Book Review: After Virtue  ·····  On virtue ethics, a reaction against modern moral philosophy.
• Read History of Philosophy Backwards  ·····  Historical texts reveal our implicit assumptions.
• Virtue Ethics: Not Practically Useful Either  ·····  Is virtue ethics useful prescriptively or descriptively?
• Last Thoughts on Virtue Ethics  ·····  What claims do virtue ethicists make?
• Proving Too Much  ·····  If an argument sometimes proves falsehoods, it can't be valid.

IX. Liberty

• The Non-Libertarian FAQ (aka Why I Hate Your Freedom)
• A Blessing in Disguise, Albeit a Very Good Disguise
• Basic Income Guarantees
• Book Review: The Nurture Assumption
• The Death of Wages is Sin
• Thank You For Doing Something Ambiguously Between Smoking And Not Smoking
• Lies, Damned Lies, and Facebook (Part 1 of ∞)
• The Life Cycle of Medical Ideas
• Vote on Values, Outsource Beliefs
• A Something Sort of Like Left-Libertarian-ist Manifesto
• Plutocracy Isn’t About Money
• Against Tulip Subsidies
• SlateStarCodex Gives a Graduation Speech

X. Progress

• Intellectual Hipsters and Meta-Contrarianism
• A Signaling Theory of Class x Politics Interaction
• Reactionary Philosophy in an Enormous, Planet-Sized Nutshell
• A Thrive/Survive Theory of the Political Spectrum
• We Wrestle Not With Flesh And Blood, But Against Powers And Principalities
• Poor Folks Do Smile… For Now
• Apart from Better Sanitation and Medicine and Education and Irrigation and Public Health and Roads and Public Order, What Has Modernity Done for Us?
• The Wisdom of the Ancients
• Can Atheists Appreciate Chesterton?
• Holocaust Good for You, Research Finds, But Frequent Taunting Causes Cancer in Rats
• Public Awareness Campaigns
• Social Psychology is a Flamethrower
• Nature is Not a Slate. It’s a Series of Levers.
• The Anti-Reactionary FAQ
• The Poor You Will Always Have With You
• Proposed Biological Explanations for Historical Trends in Crime
• Society is Fixed, Biology is Mutable

XI. Social Justice

• Practically-a-Book Review: Dying to be Free
• Drug Testing Welfare Users is a Sham, But Not for the Reasons You Think
• The Meditation on Creepiness
• The Meditation on Superweapons
• The Meditation on the War on Applause Lights
• The Meditation on Superweapons and Bingo
• An Analysis of the Formalist Account of Power Relations in Democratic Societies
• Arguments About Male Violence Prove Too Much
• Social Justice for the Highly-Demanding-of-Rigor
• Against Bravery Debates
• All Debates Are Bravery Debates
• A Comment I Posted on “What Would JT Do?”
• We Are All MsScribe
• The Spirit of the First Amendment
• A Response to Apophemi on Triggers
• Lies, Damned Lies, and Social Media: False Rape Accusations
• In Favor of Niceness, Community, and Civilization

XII. Politicization

• Right is the New Left
• Weak Men are Superweapons
• You Kant Dismiss Universalizability
• I Can Tolerate Anything Except the Outgroup
• Five Case Studies on Politicization
• Black People Less Likely
• Nydwracu’s Fnords
• All in All, Another Brick in the Motte
• Ethnic Tension and Meaningless Arguments
• Race and Justice: Much More Than You Wanted to Know
• Framing for Light Instead of Heat
• The Wonderful Thing About Triggers
• Fearful Symmetry
• Archipelago and Atomic Communitarianism

XIII. Competition and Cooperation

• Galactic Core
• Book Review: The Two-Income Trap
• Just for Stealing a Mouthful of Bread
• Meditations on Moloch
• Misperceptions on Moloch
• The Invisible Nation — Reconciling Utilitarianism and Contractualism
• Freedom on the Centralized Web
• Book Review: Singer on Marx
• Does Class Warfare Have a Free Rider Problem?
• Book Review: Red Plenty

If you liked these posts and want more, I suggest browsing the SlateStarCodex archives.

Imagine someone finding out that "Physics professors fail on basic physics problems". This, of course, would never happen. To become a physicist in academia, one has to (among million other things) demonstrate proficiency on far harder problems than that.

Philosophy professors, however, are a different story. Cosmologist Sean Carroll tweeted a link to a paper from the Harvard Moral Psychology Research Lab, which found that professional moral philosophers are no less subject to the effects of framing and order of presentation on the Trolley Problem than non-philosophers. This seems as basic an error as, say, confusing energy with momentum, or mixing up units on a physics test.

Abstract:

We examined the effects of framing and order of presentation on professional philosophers’ judgments about a moral puzzle case (the “trolley problem”) and a version of the Tversky & Kahneman “Asian disease” scenario. Professional philosophers exhibited substantial framing effects and order effects, and were no less subject to such effects than was a comparison group of non-philosopher academic participants. Framing and order effects were not reduced by a forced delay during which participants were encouraged to consider “different variants of the scenario or different ways of describing the case”. Nor were framing and order effects lower among participants reporting familiarity with the trolley problem or with loss-aversion framing effects, nor among those reporting having had a stable opinion on the issues before participating the experiment, nor among those reporting expertise on the very issues in question. Thus, for these scenario types, neither framing effects nor order effects appear to be reduced even by high levels of academic expertise.

Some quotes (emphasis mine):

When scenario pairs were presented in order AB, participants responded differently than when the same scenario pairs were presented in order BA, and the philosophers showed no less of a shift than did the comparison groups, across several types of scenario.

[...] we could find no level of philosophical expertise that reduced the size of the order effects or the framing effects on judgments of specific cases. Across the board, professional philosophers (94% with PhD’s) showed about the same size order and framing effects as similarly educated non-philosophers. Nor were order effects and framing effects reduced by assignment to a condition enforcing a delay before responding and encouraging participants to reflect on “different variants of the scenario or different ways of describing the case”. Nor were order effects any smaller for the majority of philosopher participants reporting antecedent familiarity with the issues. Nor were order effects any smaller for the minority of philosopher participants reporting expertise on the very issues under investigation. Nor were order effects any smaller for the minority of philosopher participants reporting that before participating in our experiment they had stable views about the issues under investigation.

I am confused... I assumed that an expert in moral philosophy would not fall prey to the relevant biases so easily... What is going on?

Eliezer Yudkowsky previously wrote (6 years ago!) about the second law of thermodynamics. Many commenters were skeptical about the statement, "if you know the positions and momenta of every particle in a glass of water, it is at absolute zero temperature," because they don't know what temperature is. This is a common confusion.

Entropy

To specify the precise state of a classical system, you need to know its location in phase space. For a bunch of helium atoms whizzing around in a box, phase space is the position and momentum of each helium atom. For N atoms in the box, that means 6N numbers to completely specify the system.

Lets say you know the total energy of the gas, but nothing else. It will be the case that a fantastically huge number of points in phase space will be consistent with that energy.* In the absence of any more information it is correct to assign a uniform distribution to this region of phase space. The entropy of a uniform distribution is the logarithm of the number of points, so that's that. If you also know the volume, then the number of points in phase space consistent with both the energy and volume is necessarily smaller, so the entropy is smaller.

This might be confusing to chemists, since they memorized a formula for the entropy of an ideal gas, and it's ostensibly objective. Someone with perfect knowledge of the system will calculate the same number on the right side of that equation, but to them, that number isn't the entropy. It's the entropy of the gas if you know nothing more than energy, volume, and number of particles.

Temperature

The existence of temperature follows from the zeroth and second laws of thermodynamics: thermal equilibrium is transitive, and entropy is maximum in equilibrium. Temperature is then defined as the thermodynamic quantity that is the shared by systems in equilibrium.

If two systems are in equilibrium then they cannot increase entropy by flowing energy from one to the other. That means that if we flow a tiny bit of energy from one to the other (δU₁ = -δU₂), the entropy change in the first must be the opposite of the entropy change of the second (δS₁ = -δS₂), so that the total entropy (S₁ + S₂) doesn't change. For systems in equilibrium, this leads to (∂S₁/∂U₁) = (∂S₂/∂U₂). Define 1/T = (∂S/∂U), and we are done.

Temperature is sometimes taught as, "a measure of the average kinetic energy of the particles," because for an ideal gas U/N = (3/2) k_BT. This is wrong as a definition, for the same reason that the ideal gas entropy isn't the definition of entropy.

Probability is in the mind. Entropy is a function of probabilities, so entropy is in the mind. Temperature is a derivative of entropy, so temperature is in the mind.

Second Law Trickery

With perfect knowledge of a system, it is possible to extract all of its energy as work. EY states it clearly:

So (again ignoring quantum effects for the moment), if you know the states of all the molecules in a glass of hot water, it is cold in a genuinely thermodynamic sense: you can take electricity out of it and leave behind an ice cube.

Someone who doesn't know the state of the water will observe a violation of the second law. This is allowed. Let that sink in for a minute. Jaynes calls it second law trickery, and I can't explain it better than he does, so I won't try:

A physical system always has more macroscopic degrees of freedom beyond what we control or observe, and by manipulating them a trickster can always make us see an apparent violation of the second law.

Therefore the correct statement of the second law is not that an entropy decrease is impossible in principle, or even improbable; rather that it cannot be achieved reproducibly by manipulating the macrovariables {X₁, ..., X_n} that we have chosen to define our macrostate. Any attempt to write a stronger law than this will put one at the mercy of a trickster, who can produce a violation of it.

But recognizing this should increase rather than decrease our confidence in the future of the second law, because it means that if an experimenter ever sees an apparent violation, then instead of issuing a sensational announcement, it will be more prudent to search for that unobserved degree of freedom. That is, the connection of entropy with information works both ways; seeing an apparent decrease of entropy signifies ignorance of what were the relevant macrovariables.

Homework

I've actually given you enough information on statistical mechanics to calculate an interesting system. Say you have N particles, each fixed in place to a lattice. Each particle can be in one of two states, with energies 0 and ε. Calculate and plot the entropy if you know the total energy: S(E), and then the energy as a function of temperature: E(T). This is essentially a combinatorics problem, and you may assume that N is large, so use Stirling's approximation. What you will discover should make sense using the correct definitions of entropy and temperature.

*: How many combinations of 10²³ numbers between 0 and 10 add up to 5×10²³?

This was originally part of a post I wrote on logical uncertainty, but it turned out to be post-sized itself, so I'm splitting it off.

Daniel Garber's article Old Evidence and Logical Omniscience in Bayesian Confirmation Theory. Wonderful framing of the problem--explains the relevance of logical uncertainty to the Bayesian theory of confirmation of hypotheses by evidence.

Articles on using logical uncertainty for Friendly AI theory: qmaurmann's Meditations on Löb’s theorem and probabilistic logic. Squark's Overcoming the Loebian obstacle using evidence logic. And Paul Christiano, Eliezer Yudkowsky, Paul Herreshoff, and Mihaly Barasz's Definibility of Truth in Probabilistic Logic. So8res's walkthrough of that paper, and qmaurmann's notes. eli_sennesh like just made a post on this: Logics for Mind-Building Should Have Computational Meaning.

Benja's post on using logical uncertainty for updateless decision theory.

cousin_it's Notes on logical priors from the MIRI workshop. Addresses a logical-uncertainty version of Counterfactual Mugging, but in the course of that has, well, notes on logical priors that are more general.

Reasoning with Limited Resources and Assigning Probabilities to Arithmetical Statements, by Haim Gaifman. Shows that you can give up on giving logically equivalent statements equal probabilities without much sacrifice of the elegance of your theory. Also, gives a beautifully written framing of the problem.

manfred's early post, and later sequence. Amazingly readable. The proposal gives up Gaifman's elegance, but actually goes as far as assigning probabilities to mathematical statements and using them, whereas Gaifman never follows through to solve an example afaik. The post or the sequence may be the quickest path to getting your hands dirty and trying this stuff out, though I don't think the proposal will end up being the right answer.

There's some literature on modeling a function as a stochastic process, which gives you probability distributions over its values. The information in these distributions comes from calculations of a few values of the function. One application is in optimizing a difficult-to-evaluate objective function: see Efficient Global Optimization of Expensive Black-Box Functions, by Donald R. Jones, Matthias Schonlau, and William J. Welch. Another is when you're doing simulations that have free parameters, and you want to make sure you try all the relevant combinations of parameter values: see Design and Analysis of Computer Experiments by Jerome Sacks, William J. Welch, Toby J. Mitchell, and Henry P. Wynn.

Maximize Worst Case Bayes Score, by Coscott, addresses the question: "Given a consistent but incomplete theory, how should one choose a random model of that theory?"

Bayesian Networks for Logical Reasoning by Jon Williamson. Looks interesting, but I can't summarize it because I don't understand it.

And, a big one that I'm still working through: Non-Omniscience, Probabilistic Inference, and Metamathematics, by Paul Christiano. Very thorough, goes all the way from trying to define coherent belief to trying to build usable algorithms for assigning probabilities.

Dealing With Logical Omniscience: Expressiveness and Pragmatics, by Joseph Y. Halpern and Riccardo Pucella.

Reasoning About Rational, But Not Logically Omniscient Agents, by Ho Ngoc Duc. Sorry about the paywall.

And then the references from Christiano's report:

Abram Demski. Logical prior probability. In Joscha Bach, Ben Goertzel, and Matthew Ikle, editors, AGI, volume 7716 of Lecture Notes in Computer Science, pages 50-59. Springer, 2012.

Marcus Hutter, John W. Lloyd, Kee Siong Ng, and William T. B. Uther. Probabilities on sentences in an expressive logic. CoRR, abs/1209.2620, 2012.

Bas R. Steunebrink and Jurgen Schmidhuber. A family of Godel machine implementations. In Jurgen Schmidhuber, Kristinn R. Thorisson, and Moshe Looks, editors, AGI, volume 6830 of Lecture Notes in Computer Science, pages 275{280. Springer, 2011.

If you have any more links, post them!

Or if you can contribute summaries.

This is part of a weekly reading group on Nick Bostrom's book, Superintelligence. For more information about the group, see the announcement post. For the schedule of future topics, see MIRI's reading guide.

Welcome to the Superintelligence reading group. This week we discuss the first section in the reading guide, Past developments and present capabilities. This section considers the behavior of the economy over very long time scales, and the recent history of artificial intelligence (henceforth, 'AI'). These two areas are excellent background if you want to think about large economic transitions caused by AI.

This post summarizes the section, and offers a few relevant notes, thoughts, and ideas for further investigation. My own thoughts and questions for discussion are in the comments.

There is no need to proceed in order through this post. Feel free to jump straight to the discussion. Where applicable, page numbers indicate the rough part of the chapter that is most related (not necessarily that the chapter is being cited for the specific claim).

Reading: Foreword, and Growth modes through State of the art from Chapter 1 (p1-18)

Summary

Economic growth:

1. Economic growth has become radically faster over the course of human history. (p1-2)
2. This growth has been uneven rather than continuous, perhaps corresponding to the farming and industrial revolutions. (p1-2)
3. Thus history suggests large changes in the growth rate of the economy are plausible. (p2)
4. This makes it more plausible that human-level AI will arrive and produce unprecedented levels of economic productivity.
5. Predictions of much faster growth rates might also suggest the arrival of machine intelligence, because it is hard to imagine humans - slow as they are - sustaining such a rapidly growing economy. (p2-3)
6. Thus economic history suggests that rapid growth caused by AI is more plausible than you might otherwise think.

The history of AI:

1. Human-level AI has been predicted since the 1940s. (p3-4)
2. Early predictions were often optimistic about when human-level AI would come, but rarely considered whether it would pose a risk. (p4-5)
3. AI research has been through several cycles of relative popularity and unpopularity. (p5-11)
4. By around the 1990s, 'Good Old-Fashioned Artificial Intelligence' (GOFAI) techniques based on symbol manipulation gave way to new methods such as artificial neural networks and genetic algorithms. These are widely considered more promising, in part because they are less brittle and can learn from experience more usefully. Researchers have also lately developed a better understanding of the underlying mathematical relationships between various modern approaches. (p5-11)
5. AI is very good at playing board games. (12-13)
6. AI is used in many applications today (e.g. hearing aids, route-finders, recommender systems, medical decision support systems, machine translation, face recognition, scheduling, the financial market). (p14-16)
7. In general, tasks we thought were intellectually demanding (e.g. board games) have turned out to be easy to do with AI, while tasks which seem easy to us (e.g. identifying objects) have turned out to be hard. (p14)
8. An 'optimality notion' is the combination of a rule for learning, and a rule for making decisions. Bostrom describes one of these: a kind of ideal Bayesian agent. This is impossible to actually make, but provides a useful measure for judging imperfect agents against. (p10-11)

Notes on a few things

1. What is 'superintelligence'? (p22 spoiler)
   In case you are too curious about what the topic of this book is to wait until week 3, a 'superintelligence' will soon be described as 'any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest'. Vagueness in this definition will be cleared up later. 
2. What is 'AI'?
   In particular, how does 'AI' differ from other computer software? The line is blurry, but basically AI research seeks to replicate the useful 'cognitive' functions of human brains ('cognitive' is perhaps unclear, but for instance it doesn't have to be squishy or prevent your head from imploding). Sometimes AI research tries to copy the methods used by human brains. Other times it tries to carry out the same broad functions as a human brain, perhaps better than a human brain. Russell and Norvig (p2) divide prevailing definitions of AI into four categories: 'thinking humanly', 'thinking rationally', 'acting humanly' and 'acting rationally'. For our purposes however, the distinction is probably not too important.
3. What is 'human-level' AI? 
   We are going to talk about 'human-level' AI a lot, so it would be good to be clear on what that is. Unfortunately the term is used in various ways, and often ambiguously. So we probably can't be that clear on it, but let us at least be clear on how the term is unclear. 
   
   One big ambiguity is whether you are talking about a machine that can carry out tasks as well as a human at any price, or a machine that can carry out tasks as well as a human at the price of a human. These are quite different, especially in their immediate social implications.
   
   Other ambiguities arise in how 'levels' are measured. If AI systems were to replace almost all humans in the economy, but only because they are so much cheaper - though they often do a lower quality job - are they human level? What exactly does the AI need to be human-level at? Anything you can be paid for? Anything a human is good for? Just mental tasks? Even mental tasks like daydreaming? Which or how many humans does the AI need to be the same level as? Note that in a sense most humans have been replaced in their jobs before (almost everyone used to work in farming), so if you use that metric for human-level AI, it was reached long ago, and perhaps farm machinery is human-level AI. This is probably not what we want to point at.
   
   Another thing to be aware of is the diversity of mental skills. If by 'human-level' we mean a machine that is at least as good as a human at each of these skills, then in practice the first 'human-level' machine will be much better than a human on many of those skills. It may not seem 'human-level' so much as 'very super-human'.
   
   We could instead think of human-level as closer to 'competitive with a human' - where the machine has some super-human talents and lacks some skills humans have. This is not usually used, I think because it is hard to define in a meaningful way. There are already machines for which a company is willing to pay more than a human: in this sense a microscope might be 'super-human'. There is no reason for a machine which is equal in value to a human to have the traits we are interested in talking about here, such as agency, superior cognitive abilities or the tendency to drive humans out of work and shape the future. Thus we talk about AI which is at least as good as a human, but you should beware that the predictions made about such an entity may apply before the entity is technically 'human-level'.
   
   
   Example of how the first 'human-level' AI may surpass humans in many ways.
   
   Because of these ambiguities, AI researchers are sometimes hesitant to use the term. e.g. in these interviews.
4. Growth modes (p1) 
   Robin Hanson wrote the seminal paper on this issue. Here's a figure from it, showing the step changes in growth rates. Note that both axes are logarithmic. Note also that the changes between modes don't happen overnight. According to Robin's model, we are still transitioning into the industrial era (p10 in his paper).
5. What causes these transitions between growth modes? (p1-2)
   One might be happier making predictions about future growth mode changes if one had a unifying explanation for the previous changes. As far as I know, we have no good idea of what was so special about those two periods. There are many suggested causes of the industrial revolution, but nothing uncontroversially stands out as 'twice in history' level of special. You might think the small number of datapoints would make this puzzle too hard. Remember however that there are quite a lot of negative datapoints - you need an explanation that didn't happen at all of the other times in history. 
6. Growth of growth
   It is also interesting to compare world economic growth to the total size of the world economy. For the last few thousand years, the economy seems to have grown faster more or less in proportion to it's size (see figure below). Extrapolating such a trend would lead to an infinite economy in finite time. In fact for the thousand years until 1950 such extrapolation would place an infinite economy in the late 20th Century! The time since 1950 has been strange apparently. 
   
   (Figure from here)
7. Early AI programs mentioned in the book (p5-6)
   You can see them in action: SHRDLU, Shakey, General Problem Solver (not quite in action), ELIZA.
8. Later AI programs mentioned in the book (p6)
   Algorithmically generated Beethoven, algorithmic generation of patentable inventions, artificial comedy (requires download).
9. Modern AI algorithms mentioned (p7-8, 14-15) 
   Here is a neural network doing image recognition. Here is artificial evolution of jumping and of toy cars. Here is a face detection demo that can tell you your attractiveness (apparently not reliably), happiness, age, gender, and which celebrity it mistakes you for.
10. What is maximum likelihood estimation? (p9)
    Bostrom points out that many types of artificial neural network can be viewed as classifiers that perform 'maximum likelihood estimation'. If you haven't come across this term before, the idea is to find the situation that would make your observations most probable. For instance, suppose a person writes to you and tells you that you have won a car. The situation that would have made this scenario most probable is the one where you have won a car, since in that case you are almost guaranteed to be told about it. Note that this doesn't imply that you should think you won a car, if someone tells you that. Being the target of a spam email might only give you a low probability of being told that you have won a car (a spam email may instead advise you of products, or tell you that you have won a boat), but spam emails are so much more common than actually winning cars that most of the time if you get such an email, you will not have won a car. If you would like a better intuition for maximum likelihood estimation, Wolfram Alpha has several demonstrations (requires free download).
11. What are hill climbing algorithms like? (p9)
    The second large class of algorithms Bostrom mentions are hill climbing algorithms. The idea here is fairly straightforward, but if you would like a better basic intuition for what hill climbing looks like, Wolfram Alpha has a demonstration to play with (requires free download).

In-depth investigations

If you are particularly interested in these topics, and want to do further research, these are a few plausible directions:

1. How have investments into AI changed over time? Here's a start, estimating the size of the field.
2. What does progress in AI look like in more detail? What can we infer from it? I wrote about algorithmic improvement curves before. If you are interested in plausible next steps here, ask me.
3. What do economic models tell us about the consequences of human-level AI? Here is some such thinking; Eliezer Yudkowsky has written at length about his request for more.

How to proceed

This has been a collection of notes on the chapter. The most important part of the reading group though is discussion, which is in the comments section. I pose some questions for you there, and I invite you to add your own. Please remember that this group contains a variety of levels of expertise: if a line of discussion seems too basic or too incomprehensible, look around for one that suits you better!

Next week, we will talk about what AI researchers think about human-level AI: when it will arrive, what it will be like, and what the consequences will be. To prepare, read Opinions about the future of machine intelligence from Chapter 1 and also When Will AI Be Created? by Luke Muehlhauser. The discussion will go live at 6pm Pacific time next Monday 22 September. Sign up to be notified here.

This is crossposted from my new blog. I was planning to write a short post explaining how Newcomblike problems are the norm and why any sufficiently powerful intelligence built to use causal decision theory would self-modify to stop using causal decision theory in short order. Turns out it's not such a short topic, and it's turning into a short intro to decision theory.

I've been motivating MIRI's technical agenda (decision theory and otherwise) to outsiders quite frequently recently, and I received a few comments of the form "Oh cool, I've seen lots of decision theory type stuff on LessWrong, but I hadn't understood the connection." While the intended audience of my blog is wider than the readerbase of LW (and thus, the tone might seem off and the content a bit basic), I've updated towards these posts being useful here. I also hope that some of you will correct my mistakes!

This sequence will probably run for four or five posts, during which I'll motivate the use of decision theory, the problems with the modern standard of decision theory (CDT), and some of the reasons why these problems are an FAI concern.

I'll be giving a talk on the material from this sequence at Purdue next week.

1

Choice is a crucial component of reasoning. Given a set of available actions, which action do you take? Do you go out to the movies or stay in with a book? Do you capture the bishop or fork the king? Somehow, we must reason about our options and choose the best one.

Of course, we humans don't consciously weigh all of our actions. Many of our choices are made subconsciously. (Which letter will I type next? When will I get a drink of water?) Yet even if the choices are made by subconscious heuristics, they must be made somehow.

In practice, decisions are often made on autopilot. We don't weigh every available alternative when it's time to prepare for work in the morning, we just pattern-match the situation and carry out some routine. This is a shortcut that saves time and cognitive energy. Yet, no matter how much we stick to routines, we still spend some of our time making hard choices, weighing alternatives, and predicting which available action will serve us best.

The study of how to make these sorts of decisions is known as Decision Theory. This field of research is closely intertwined with Economics, Philosophy, Mathematics, and (of course) Game Theory. It will be the subject of today's post.

In just over two weeks I will be running an online reading group on Nick Bostrom's Superintelligence, on behalf of MIRI. It will be here on LessWrong. This is an advance warning, so you can get a copy and get ready for some stimulating discussion. MIRI's post, appended below, gives the details.

Added: At the bottom of this post is a list of the discussion posts so far.

Nick Bostrom’s eagerly awaited Superintelligence comes out in the US this week. To help you get the most out of it, MIRI is running an online reading group where you can join with others to ask questions, discuss ideas, and probe the arguments more deeply.

The reading group will “meet” on a weekly post on the LessWrong discussion forum. For each ‘meeting’, we will read about half a chapter of Superintelligence, then come together virtually to discuss. I’ll summarize the chapter, and offer a few relevant notes, thoughts, and ideas for further investigation. (My notes will also be used as the source material for the final reading guide for the book.)

Discussion will take place in the comments. I’ll offer some questions, and invite you to bring your own, as well as thoughts, criticisms and suggestions for interesting related material. Your contributions to the reading group might also (with permission) be used in our final reading guide for the book.

We welcome both newcomers and veterans on the topic. Content will aim to be intelligible to a wide audience, and topics will range from novice to expert level. All levels of time commitment are welcome.

We will follow this preliminary reading guide, produced by MIRI, reading one section per week.

If you have already read the book, don’t worry! To the extent you remember what it says, your superior expertise will only be a bonus. To the extent you don’t remember what it says, now is a good time for a review! If you don’t have time to read the book, but still want to participate, you are also welcome to join in. I will provide summaries, and many things will have page numbers, in case you want to skip to the relevant parts.

If this sounds good to you, first grab a copy of Superintelligence. You may also want to sign up here to be emailed when the discussion begins each week. The first virtual meeting (forum post) will go live at 6pm Pacific on Monday, September 15th. Following meetings will start at 6pm every Monday, so if you’d like to coordinate for quick fire discussion with others, put that into your calendar. If you prefer flexibility, come by any time! And remember that if there are any people you would especially enjoy discussing Superintelligence with, link them to this post!

Topics for the first week will include impressive displays of artificial intelligence, why computers play board games so well, and what a reasonable person should infer from the agricultural and industrial revolutions.

Posts in this sequence

Week 1: Past developments and present capabilities

Week 2: Forecasting AI

Week 3: AI and uploads

Week 4: Biological cognition, BCIs, organizations

Week 5: Forms of superintelligence

Week 6: Intelligence explosion kinetics

Week 7: Decisive strategic advantage

Week 8: Cognitive superpowers

Week 9: The orthogonality of intelligence and goals

Week 10: Instrumentally convergent goals

Week 11: The treacherous turn

Week 12: Malignant failure modes

Week 13: Capability control methods

Week 14: Motivation selection methods

Week 15: Oracles, genies and sovereigns

Week 16: Tool AIs

Week 17: Multipolar scenarios

Week 18: Life in an algorithmic economy

Week 19: Post-transition formation of a singleton

Week 20: The value-loading problem

Week 21: Value learning

Week 22: Emulation modulation and institution design

Week 23: Coherent extrapolated volition

Week 24: Morality models and "do what I mean"

Week 25: Components list for acquiring values

Week 26: Science and technology strategy

Week 27: Pathways and enablers

Week 28: Collaboration

Week 29: Crunch time

All of these things I mentioned in the most recent open thread, but since the first one is directly relevant and the comment where I posted it somewhat hard to come across, I figured I'd make a post too.

Custom Comment Highlights

NOTE FOR FIREFOX USERS: this contained a bug which has been squashed, causing the list of comments not to be automatically populated (depending on your version of Firefox). I suggest reinstalling. Sorry, no automatic updates unless you use the Chrome extension (though with >50% probability there will be no further updates).

You know how the highlight for new comments on Less Wrong threads disappears if you reload the page, making it difficult to find those comments again? Here is a userscript you can install to fix that (provided you're on Firefox or Chrome). Once installed, you can set the date after which comments are highlighted, and easily scroll to new comments. See screenshots. Installation is straightforward (especially for Chrome, since I made an extension as well).

Bonus: works even if you're logged out or don't have an account, though you'll have to set the highlight time manually.

Delay Before Commenting

Slate Star Codex Comment Highlighter

Note for LW Admins / Yvain