Followup to:  Fighting Akrasia:  Incentivising Action

Influenced by:  Generalizing From One Example

Previously I looked at how we might fight akrasia by creating incentives for actions.  Based on the comments to the previous article and Yvain's now classic post Generalizing From One Example, I want to take a deeper look at the source of akrasia and the techniques used to fight it.

I feel foolish for not looking at this closer first, but let's begin by asking what akrasia is and what causes it.  As commonly used, akrasia is the weakness-of-will we feel when we desire to do something but find ourselves doing something else.  So why do we experience akrasia?  Or, more to the point, why to we feel a desire to take actions contrary the actions we desire most, as indicated by our actions?  Or, if it helps, flip that question and ask why are the actions we take not always the ones we feel the greatest desire for?

First, we don't know the fine details of how the human brain makes decisions.  We know what it feels like to come to a decision about an action (or anything else), but how the algorithm feels from the inside is not a reliable way to figure out how the decision was actually made.  But because most people can relate to a feeling of akrasia, this suggests that there is some disconnect between how the brain decides what actions are most desirable and what actions we believe are most desirable.  The hypothesis that I consider most likely is that the ability to form beliefs about desirable actions evolved well after the ability to make decisions about what actions are most desirable, and the decision-making part of the brain only bothers to consult the belief-about-desirability-of-actions part of the brain when there is a reason to do so from evolution's point of view.¹  As a result we end up with a brain that only does what we think we really want when evolutionarily prudent, hence we experience akrasia whenever our brain doesn't consider it appropriate to consult what we experience as desirable.

This suggests two main ways of overcoming akrasia assuming my hypothesis (or something close to it) is correct:  make the actions we believe to be desirable also desirable to the decision-making part of the brain or make the decision-making part of the brain consult the belief-about-desirability-of-actions part of the brain when we want it to.  Most techniques fall into the former category since this is by far the easier strategy, but however a technique works, an overriding theme of the akrasia-related articles and comments on Less Wrong is that no technique yet found seems to work for all people.

This has been in my drafts folder for ages, but in light of Eliezer's post yesterday, I thought I'd see if I could get some comment on it:

A couple weeks ago, Vladimir Nesov stirred up the biggest hornet's nest I've ever seen on LW by introducing us to the Counterfactual Mugging scenario.

If you didn't read it the first time, please do -- I don't plan to attempt to summarize.  Further, if you don't think you would give Omega the $100 in that situation, I'm afraid this article will mean next to nothing to you.

So, those still reading, you would give Omega the $100.  You would do so because if someone told you about the problem now, you could do the expected utility calculation 0.5*U(-$100)+0.5*U(+$10000)>0.  Ah, but where did the 0.5s come from in your calculation?  Well, Omega told you he flipped a fair coin.  Until he did, there existed a 0.5 probability of either outcome.  Thus, for you, hearing about the problem, there is a 0.5 probability of your encountering the problem as stated, and a 0.5 probability of your encountering the corresponding situation, in which Omega either hands you $10000 or doesn't, based on his prediction.  This is all very fine and rational.  

So, new problem.  Let's leave money out of it, and assume Omega hands you 1000 utilons in one case, and asks for them in the other -- exactly equal utility.  What if there is an urn, and it contains either a red or a blue marble, and Omega looks, maybe gives you the utility if the marble is red, and asks for it if the marble is blue?  What if you have devoted considerable time to determining whether the marble is red or blue, and your subjective probability has fluctuated over the course of you life? What if, unbeknownst to you, a rationalist community has been tracking evidence of the marble's color (including your own probability estimates), and running a prediction market, and Omega now shows you a plot of the prices over the past few years?

In short, what information do you use to calculate the probability you plug into the EU calculation?

Related to: Counterfactual Mugging, The Least Convenient Possible World

MBlume said:

What would you do in situation X?" and "What would you like to pre-commit to doing, should you ever encounter situation X?" should, to a rational agent, be one and the same question.

Applied to Vladimir Nesov's counterfactual mugging, the reasoning is then:

Precommitting to paying $100 to Omega has expected utility of $4950.p(Omega appears). Not precommitting has strictly less utility; therefore I should precommit to paying. Therefore I should, in fact, pay $100 in the event (Omega appears, coin is tails).

To combat the argument that it is more likely that one is insane than that Omega has appeared, Eliezer said:

So imagine yourself in the most inconvenient possible world where Omega is a known feature of the environment and has long been seen to follow through on promises of this type; it does not particularly occur to you or anyone that believing this fact makes you insane.

My first reaction was that it is simply not rational to give $100 away when nothing can possibly happen in consequence. I still believe that, with a small modification: I believe, with moderately high probability, that it will not be instrumentally rational for my future self to do so. Read on for the explanation.