Vladimir_Nesov comments on Dissolving the Question - Less Wrong

44 Post author: Eliezer_Yudkowsky 08 March 2008 03:17AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (104)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: Sebastian_Hagen2 09 March 2008 12:49:14PM 5 points [-]

I think I'll give this a try. Let's start with what a simple non-introspective mind might do:

Init (probably recomputed sometimes, but cached most of the time): I1. Draws a border around itself, separating itself from the "outside world" in its world model. In humans and similarly embodied intelligences you could get away with defining the own body as "inside", if internal muscle control works completely without inspection.

Whenever deciding on what to output: A1. Generates a list of all possible next actions of itself, as determined in I1. For human-like embodieds, could be a list of available body movements. A2. Computes a probability distribution about the likely future states of the world resulting from each action, consulting the internal world-model for prediction. Resolution, temporal range and world-model building are beyond the scope of this answer. A3. Assigns utilities to each considered future state. A4. Assigns preferences to the probability distribution of futures. This could e.g. use Expected Utility or some satisficing algorithm. A5. Chooses the possible next action with the most-prefered distribution of futures. Tie breaking is implementation defined. A6. Execute that action.

As part of A2, the reactions of other intelligent entites are modeled as part of the general world model. This kind of mind architecture does not model itself in the present; that'd lead to infinite recursion: "I'm thinking about myself thinking about myself thinking about ...". It also wouldn't achieve anything, since the mind as instantiated necessarily has a higher resolution than any model of itself stored inside itself. It will, however, model past (for lost data) or future versions of itself.

The important point here is that this mind doesn't model itself while computing the next action. In the extreme case it needn't have facilities for introspection at all. Humans obviously have some such facilities. Either inbetween deciding on output, inbetween the individual steps A1-A6, completely in parallel, or some combination of those, humans spend cputime to analyze the algorithm they're executing, to determine systematic errors or possibilities for optimization. I'll call the neural module / program that does this the introspector. When introspecting a thread which generates motor output (A1-A6 above), one (very effective) assumption of the introspected algorithm will always turn out to be "My own next action is to-be-determined. It'd be ineffective for me to run a model of myself to determine it.". For a mind that doesn't intuitively understand the self-reference and infinite-recursion parts, this turns into "My own actions can't be modeled in advance. I have free will.".

In the cases where the introspected thread is running another instance of the introspector, the introspector still isn't attached to its own thread; doing that would lead to infinite recursion. Each introspector will work similarly to the motor-output-choosing algorithm described above, except that the generated output will be in the form of new mental heuristics. Therefore, the same "It'd be ineffective to run a model of myself to determine my next action." assumption in the algorithm can be observed, and "Free will." is still the likely conclusion of a mind that doesn't understand the rationale behind the design.

Comment author: Endovior 30 March 2011 05:44:56PM 1 point [-]

That's a model along the lines of the one I was thinking of in response to the question; any number of simple algorithms for processing data, creating a worldview, determining the expected utility of a series of actions, and choosing the action which seems to have the greatest utility might believe it has 'free will', by the definition that its actions cannot be predicted, if it is not capable of understanding its own nature.

Humans are, of course, more complicated than this, but idea alone produces the question... is your mind the sort of algorithm which, if all of its processing details, utility functions, and available worldview data are fully known, will produce output which can be predicted in advance, given the information? That doesn't feel like an ending, but it is, perhaps, grounds to explore further.

Comment author: Endovior 08 April 2011 11:35:18PM 4 points [-]

Following up...

Having (almost) finished the Quantum Physics sequence since this last comment, and come to the point at which this particular assignment is referred to, I figured I'd post my final conclusion here before 'looking at the answer', as it were.

Given a basic understanding of QM, and further understanding that macroscopic phenomenon are an extension of those same principles... Knowing that nothing 'epiphenomenal' is relevant to the question of consciousness... And assuming that no previously unobserved micro-phenomenon is responsible for consciousness, by virtue of the fact that even if there were, there is, at present, no reason to privilege that particular hypothesis...

There's no question left. What we call consciousness is simply our view of the algorithm from the inside. I believe that I have free will because it seems like the choices I make change the future I find myself in... but there are a great many other factors implicit in my thinking before I even arrive at the point of making a choice, and the fact that the probabilities involved in defining that future are impossible to calculate under existing technology does not mean that such a feat will never be possible.

That said, even full knowledge of the state of a given brain would not allow you to predict it's output in advance, as even in isolation, that brain would divide into a very large number of possible states every instant, and QM proves that there is no way of determining, in advance, which state that brain will arrive at at any given time. This is not randomness, per se... given sufficient information, and isolation from contaminating entanglements, one could theoretically plot out a map of possible states, and assign probabilities to them, and have reasonable expectations of finding that mind in the predicted states after a determined time... but could never be certain of finding any given result after any amount of time.

That doesn't mean that I don't have control over my actions, or that my perception of consciousness is an illusion... what it does mean is that I run on the same laws of physics as anything else, and the algorithms that comprise my awareness are not specially privileged to ignore or supersede those laws. Realizing this fact is no reason to do anything drastic or strange... this is the way that things have been all along, and my acknowledgment of it doesn't detract from the reality of my experiences. I could believe that my actions are determined by chance instead of choice, but that would neither be useful, nor entirely true. Whatever the factors that go into the operation of my cognitive algorithms, they ultimately add up to me. Given this, I might still believe that I have free will... while at the same time knowing that the question itself does not have the meaning I thought it had before I seriously considered the question.

Comment author: Vladimir_Nesov 04 January 2012 07:07:15PM *  0 points [-]

That said, even full knowledge of the state of a given brain would not allow you to predict it's output in advance, as even in isolation, that brain would divide into a very large number of possible states every instant, and QM proves that there is no way of determining, in advance, which state that brain will arrive at at any given time.

This is not a problem. A computer runs a program the same way in almost all future outcomes, weighed by probability. QM shows how to determine what happens even in cases where it's not as simple as that.