I'm participating in a university course on free will. On the online forum, someone asked me to summarise Eliezer's solution to the free will problem, and I did it like this. Is it accurate in this form? How should I change it?

“I'll try to summarise Yudkowsky's argument.

As Anneke pointed out, it's kinda difficult to decide what the concept of free will means. How would particles or humans behave differently if they had free will compared to if they didn't? It doesn't seem like our argument is about what we actually expect to see happening.

This is similar to arguing about whether a tree falling in a deserted forest makes any noise. If two people are arguing about this, they probably agree that if we put a microphone in the forest, it would pick up vibrations. And they also agree that no-one is having the sense experience of hearing the tree fall. So they're arguing over what 'sound' means. Yudkowsky proposes a psychological reason why people may have that particular confusion, based on how human brains work.

So with respect to free will, we can instead ask the question, “Why would humans feel like they have free will?” If we can answer this well enough, then hopefully we can dissolve the original question.

It feels like I choose between some of my possible futures. I can imagine waking up tomorrow and going to my Engineering lecture, or staying in my room and using Facebook. Both of those imaginings feel equally 'possible'.

Humans execute a decision making algorithm which is fairly similar to the following one.

1. List all your possible actions. For my lecture example, that was “Go to lecture” and “Stay home.”

2. Predict the state of the universe after pretending that you will take each possible action. We end up with “Buck has learnt stuff but not Facebooked” and “Buck has not learnt stuff but has Facebooked.”

3. Decide which is your favourite outcome. In this case, I'd rather have learnt stuff. So that's option 2.

4. Execute the action associated with the best outcome. In this case, I'd go to my lecture.

Note that the above algorithm can be made more complex and powerful, for example by incorporating probability and quantifying your preferences as a utility function.

As humans, our brains need the capacity to pretend that we could choose different things, so that we can imagine the outcomes, and pick effectively. The way our brain implements this is by considering those possible worlds which we could reach through our choices, and by treating them as possible.

So now we have a fairly convincing explanation of why it would feel like we have free will, or the ability to choose between various actions: it's how our decision making algorithm feels from the inside.”

I just came back from the film Limitless. The movie contained a very interesting depiction of a character who gets an black market nootropic that works very well. It gives him perfect recall, perfect situational awareness, and the ability to figure out the best thing to say/do in real time. He uses this new power to finish his book, get back together with his girlfriend, become rich, and eventually become president of the united states. Incidentally he gets in shape, establishes himself as high status at top tier social events, learns many new languages, and sleeps with a bunch of women. In the end his new found intelligence leads him to happiness. The drama in the film comes from the fact that the drug has side effects, and there's a mobster who gets his hands on some and wants more. Intelligence is depicted as a fundamentally good thing. It's even described as "I knew what I wanted and I knew how to get it." He affirms several times (and the story agrees) that he's still himself on the drug, just more effective. 

Related to: Cached selves, Why you're stuck in a narrative, The curse of identity

Outline: Some back-story, Pondering the mechanics of self-image, The role of narrative, Narrative as a medium for self-communication.

tl;dr: One can have a self-image that causes one to neglect the effects of self-image. And, since we tend to process our self-images somewhat in the context of a narrative identity, if you currently make zero use of narrative in understanding and affecting how you think about yourself, it may be worth adjusting upward. All this seems to have been the case for me, and is probably part of what makes HPMOR valuable.

Some back-story

Starting when I was around 16 and becoming acutely annoyed with essentialism, I prided myself on not being dependent on a story-like image of myself. In fact, to make sure I wasn't, I put a break command in my narrative loop: I drafted a story in my mind about a hero who was able to outwit his foes by being less constrained by narrative than they were, and I identified with him whenever I felt a need-for-narrative coming on. Batman's narrator goes for something like this in the Dark Knight when he <select for spoiler-> abandons his heroic image to take the blame for Harvey Dent's death.

I think this break command was mostly a good thing. It helped me to resolve cognitive dissonance and overcome the limitations of various cached selves, and I ended up mostly focussed on whether my beliefs were accurate and my desires were being fulfilled. So I still figure it's a decent first-order correction to being over-constrained by narrative.

But, I no longer think it's the only decent solution. In fact, understanding the more subtle mechanics of self-image — what affects our self schemas, what they affect, and how — was something I neglected for a long time because I saw self-image as a solved problem. Yes, I developed a cached view of myself as unaffected by self-image constraints. I would have been embarassed to notice such dependencies, so I didn't. The irony, eh?

I'm writing this because I wouldn't be surprised to find others here developing, or having developed, this blind spot...

Suppose it turned out that humans violate the axioms of VNM rationality (and therefore don't act like they have utility functions) because there are three valuation systems in the brain that make conflicting valuations, and all three systems contribute to choice. And suppose that upon reflection we would clearly reject the outputs of two of these systems, whereas the third system looks something more like a utility function we might be able to use in CEV.

What I just described is part of the leading theory of choice in the human brain.

Recall that human choices are made when certain populations of neurons encode expected subjective value (in their firing rates) for each option in the choice set, with the final choice being made by an argmax or reservation price mechanism.

Today's news is that our best current theory of human choices says that at least three different systems compute "values" that are then fed into the final choice circuit:

• The model-based system "uses experience in the environment to learn a model of the transition distribution, outcomes and motivationally-sensitive utilities." (See Sutton & Barto 1998 for the meanings of these terms in reinforcement learning theory.) The model-based system also "infers choices by... building and evaluating the search decision tree to work out the optimal course of action." In short, the model-based system is responsible for goal-directed behavior. However, making all choices with a goal-directed system using something like a utility function would be computationally prohibitive (Daw et al. 2005), so many animals (including humans) first evolved much simpler methods for calculating the subjective values of options (see below).

• The model-free system also learns a model of the transition distribution and outcomes from experience, but "it does so by caching and then recalling the results of experience rather than building and searching the tree of possibilities. Thus, the model-free controller does not even represent the outcomes... that underlie the utilities, and is therefore not in any position to change the estimate of its values if the motivational state changes. Consider, for instance, the case that after a subject has been taught to press a lever to get some cheese, the cheese is poisoned, so it is no longer worth eating. The model-free system would learn the utility of pressing the lever, but would not have the informational wherewithal to realize that this utility had changed when the cheese had been poisoned. Thus it would continue to insist upon pressing the lever. This is an example of motivational insensitivity."

• The Pavlovian system, in contrast, calculates values based on a set of hard-wired preparatory and consummatory "preferences." Rather than calculate value based on what is likely to lead to rewarding and punishing outcomes, the Pavlovian system calculates values consistent with automatic approach toward appetitive stimuli, and automatic withdrawal from aversive stimuli. Thus, "animals cannot help but approach (rather than run away from) a source of food, even if the experimenter has cruelly arranged things in a looking-glass world so that the approach appears to make the food recede, whereas retreating would make the food more accessible (Hershberger 1986)."

Or, as Jandila put it:

• Model-based system: Figure out what's going on, and what actions maximize returns, and do them.
• Model-free system: Do the thingy that worked before again!
• Pavlovian system: Avoid the unpleasant thing and go to the pleasant thing. Repeat as necessary.

I just watched this, a very pretty version of "don't try to make yourself different, just accept who you are", and I realized that self-directed change in fiction is a worthwhile topic.

What I'm looking for is stories where main characters change themselves in ways which are basically improvements-- getting beyond the usual human is a plus, but for purposes of this discussion I'm including any significant positive change.

Another big plus would be the character needing to learn which of their goals make sense, and which methods work.

**ETA:** That was a bit of a stub-- HPMOR is partly about Harry and Hermione changing themselves, generally for the better I think (I've only read it once). It would be interesting to see what happens if Quirrell decides he needs to upgrade himself.

*Stranger in a Strange Land* is an interesting partial example-- the Martian language is presumably an upgrade for the human race, but it was developed by and for Martians, and needs some modification.

A *lot* of relatively recent fiction has people learning martial arts. I think appearance makeovers (typically for women) have become less common. I don't think there's a lot of fiction about appearance makeovers for men-- *The Stars My Destination* has one, but it's offstage. It wouldn't surprise me if *The Count of Monte Cristo* (frequently referenced with TSMD) has one.

Among the goals of Less Wrong is to "raise the sanity waterline" of humanity. We've also talked about "raising the rationality waterline": the phrase is somewhat popular around these parts, which suggests that the metaphor is catchy. But is that all there is to it, a catchy metaphor? Or can the phrase be more usefully cashed out?

While reading Nate Silver's The Signal and the Noise, I came across a discussion of "raising the waterline" which fleshes out the metaphor with a more substantial model. This model preserves some of the salient aspects of the metaphor as discussed on LW, for instance the perception that the current waterline (as regards sanity and rationality) is "ridiculously low". More interestingly, it fleshes out some of the specific ways that a "waterline" belief should constrain our future sensory experiences, maybe even to the point of quantifying what should result from low (or rising) waterlines.

This is intended as a short series:

• "Raising the waterline", this introductory post, will summarize Nate Silver's "waterline" model, within its original context of playing Poker, which Silver frames as a game of prediction under uncertainty. Poker therefore serves as a "toy model" for a much more general class of problems.
• "Raising the forecasting waterline" will extend the discussion to the kind of forecasts studied by Philip Tetlock's Good Judgement Project, a prediction game somewhat similar to PredictionBook and related to prediction markets; I will leverage the waterline model to extract useful insights from my participation in GJP.
• "Raising the discussion waterline", a shamelessly speculative coda, will relate the previous two posts to the question of "how do Internet discussions reliably lead to correct inferences from true beliefs, or fail to do so"; I will argue that the waterline model brings some hope that a few basic tactics could nevertheless provide large wins, and raise the more general question of what other low waterlines we could aim to exploit.