Epistemic Status: confused & unlikely

Author's note: the central claim of this article I now believe is confused, and mostly inaccurate. More precisely (in response to a comment by ChristianKl)

>Whose idea of reductionism are you criticising? I think your post could get more useful by being more clear about the idea you want to challenge.

I think this is closest I get to having a "Definiton 3.4.1" in my post

"...the other reductionism I mentioned, the 'big thing = small thing + small thing' one..."

Essentially, the claim is that to accurately explain reality, non-reductionist explanations aren't always *wrong*. 

The confusion, however, that I realized elsewhere in the thread, is that I conflate 'historical explanation' with 'predictive explanation'. Good predictive explanation will almost always be reductionist, because, as it says on the tin, big are made of smaller things. Good historical explanations, though, will be contra-reductionist, they'll explain phenomena in terms of its relation to the environment. Consider evolution; the genes seem to be explained non-reductionistically because their presence or absence is determined by its effect on the environment i.e. whether its fit, so the explanation for how it got there necessarily includes complex things because they cause it.

>Apart from that I don't know what you mean with theory in "Reductionism is a philosophy, not a theory." As a result on using a bunch of terms where I don't know exactly what you mean it's hard to follow your argument.

Artifact of confusion;  if contra-reductionism is a valid platform for explanation, then the value of reductionism isn't constative -- that is, it isn't about whether it's true or false, but something at the meta-level, rather than the object level. The antecedent is no longer believed, so now I do not believe the consequent.

The conceit I had by calling it a philosophy, or more accurately, a perspective, is essentially that you have a dataset, then you can apply a 'reductionist' filter on it to get reductionist explanations and a 'contra-reductionist' filter to get contra explanations. This was a confusion; and only seemed reasonable because I I was treating the two type of explanation -- historical and predictive -- as somehow equivalent, which I now know to be mistaken.

Reductionism is usually thought of as the assertion that the sum of the parts equal the whole. Or, a bit more polemically, that reductionist explanations more meaningful, proper, or [insert descriptor laced with postive affect]. It's certainly appealing, you could even say it seems reality prefers these types of explanation. The facts of biology can be attributed to the effects of chemistry, the reactions of chemistry can be attributed to the interplay of atoms, and so on.

But this is conflating what is seen with the perspective itself; I see a jelly donut therefore I am a jelly donut is not a valid inference. Reductionism is a way of thinking about facts, but it is not the facts themselves. Reductionism is a philosophy, not a theory. The closest thing to an testable prediction it makes it what could be termed an anti-prediction.

Another confusion concerns the alternatives to reductionism. The salient instance of anti-reduction tends to be some holist quantum spirituality woo, but I contend this is more of a weak man than anything. To alleviate any confusion, I'll just refer to my proposed notion as 'contra-reductionism'.

Earlier, I mentioned reductionism makes no meaningful predictions. To clarify this, I'll distinguish from a kind a diminutive motte of reductionism which may or may not actually exist outside my own mind, (and which truly is just a species of causality, broadly construed). In broad strokes, this reductionism 'reduces' a phenomena to the sum of it's causes, as opposed to its parts. This is the kind of reductionist explanation that treats evolution as a reductionist explanation, indeed it treats almost any model which isn't strictly random as 'reductionist'. The other referent would be reductionism as the belief that "big things are made of a smaller things, and complex things are made of simpler things". 

It's is the former kind of reductionism that makes what I labeled an anti-prediction, the core of this argument is simply that reductionist is about causality; specifically, it qualifies what types of causes should even be considered meaningful or well-founded or simply, worth thinking about. If you broaden the net sufficiently, causality is a concept which even makes sense to apply to mathematical abstraction completely unrooted in any kind of time. That is the interventionist account of causality essentially boils it down to 'what levers could we have pulled to make something not happen', which perfectly translates to maths, see, for instance, reductio ad absurdum arguments.

But I digress. This diminutive reductionism here is simply the belief that things can be reduced to their causes, which is on par with defining transhumanism as 'simplified humanism' in the category of useless philosophical mottes. In short, this is quite literally an assertion of no substance, and isn't even worth giving a name.

Now that I've finished attacking straw men, the other reductionism I mentioned, the 'big thing = small thing + small thing' one, is also flawed, albeit useful nonetheless.

This can be illustrated by the example of evolution I mentioned: An evolutionary explanation is actually anti-reductionist; it explains the placement of nucleotides in terms of mathematics like inclusive genetic fitness and complexities like population ecology. Put bluntly, the there is little object-level difference between explaining genes sequences with evolution and explaining weather with pantheons of gods (there is meta-level difference; i.e. one is accurate). Put less controversially, this is explicitly non-reductionistic; relatively simple things (the genetic sequence of a creature) are explained in the language of things far more complex (population and environment dynamics over the course of billions of years). If this is your reductionism, all it does is encapsulate the ontology of universe-space, or more evocatively, it's a logic that doesn't -- couldn't -- tell you where you live, because doesn't change wherever you may go.

Another situation where reductionism  and contra-reductionism give different answers is an example cribbed from David Deutsch. It's possible to set up dominos so that they compute an algorithm which decides the primality of 631. How would you explain a a positive result?

The reductionist explanation is approximately: "the domino remains standing because the one behind it didn't fall over", and so on with variation such as "that domino didn't fall over because the one behind it was knockovered sideways". The contra-reductionist explanation is "that domino didn't fall over "because 631 is prime". Each one is 'useful' depending on whether you are concerned with the mechanics of the domino computer or the theory.

You might detect something in these passages -- that while I slough off any pretense of reductionism, glorious (philosophical) materialism remains a kind of true north in my analysis. This is my thesis. My contra-reductionism isn't non-materialistic, it's merely a perspective inversion of the sort highlighted by a figure/ground illusion. Reductionism defines -- reduces -- objects by pointing to their constituents. A mechanism functions because its components function. A big thing of small things. Quasi-reductionism  does the opposite, it defined objects by their impact on other objects, "[A] tree is only a tree in the shade it gives to the ground below, to the relationship of wind to branch and air to leaf." I don't mean this in a spiritual way, naturally (no pun intended). I am merely defining objects externally rather than internally. At the core, the rose is still a rose, the sum is still normality.

If I had to give a short, pithy summation of this post, the core is simply that, like all systematized notions of truth or meaningfulness, reductionism collapses in degenerate cases where it fails to be useful or give the right answer. Contra-reductionism isn't a improvement or a replacement, but a alternative formulation in a conceptual monoculture, which happens to give right answer sometimes.

In the Mary's room thought experiment, Mary is a brilliant scientist in a black-and-white room who has never seen any colour. She can investigate the outside world through a black-and-white television, and has piles of textbooks on physics, optics, the eye, and the brain (and everything else of relevance to her condition). Through this she knows everything intellectually there is to know about colours and how humans react to them, but she hasn't seen any colours at all.

After that, when she steps out of the room and sees red (or blue), does she learn anything? It seems that she does. Even if she doesn't technically learn something, she experiences things she hadn't ever before, and her brain certainly changes in new ways.

The argument was intended as a defence of qualia against certain forms of materialism. It's interesting, and I don't intent to solve it fully here. But just like I extended Searle's Chinese room argument from the perspective of an AI, it seems this argument can also be considered from an AI's perspective.

Consider a RL agent with a reward channel, but which currently receives nothing from that channel. The agent can know everything there is to know about itself and the world. It can know about all sorts of other RL agents, and their reward channels. It can observe them getting their own rewards. Maybe it could even interrupt or increase their rewards. But, all this knowledge will not get it any reward. As long as its own channel doesn't send it the signal, knowledge of other agents rewards - even of identical agents getting rewards - does not give this agent any reward. Ceci n'est pas une récompense.

This seems to mirror Mary's situation quite well - knowing everything about the world is no substitute from actually getting the reward/seeing red. Now, a RL's agent reward seems closer to pleasure than qualia - this would correspond to a Mary brought up in a puritanical, pleasure-hating environment.

Closer to the original experiment, we could imagine the AI is programmed to enter into certain specific subroutines, when presented with certain stimuli. The only way for the AI to start these subroutines, is if the stimuli is presented to them. Then, upon seeing red, the AI enters a completely new mental state, with new subroutines. The AI could know everything about its programming, and about the stimulus, and, intellectually, what would change about itself if it saw red. But until it did, it would not enter that mental state.

If we use ⬜ to (informally) denote "knowing all about", then ⬜(X→Y) does not imply Y. Here X and Y could be "seeing red" and "the mental experience of seeing red". I could have simplified that by saying that ⬜Y does not imply Y. Knowing about a mental state, even perfectly, does not put you in that mental state.

This closely resembles the original Mary's room experiment. And it seems that if anyone insists that certain features are necessary to the intuition behind Mary's room, then these features could be added to this model as well.

Mary's room is fascinating, but it doesn't seem to be talking about humans exclusively, or even about conscious entities.

(Nothing here is actually new, but a short explanation with pictures would have been helpful to me a while ago, so I thought I'd make an attempt.)

Let me start with a patch of territory: a set of things that exist. The number of rows is far from clear, but I'll use six candidates as a sample; and of course the diagram ought to be a tree, with many elements on each row converging to fewer on the row above, but you'll have to imagine that part.



The blue line that runs through the column is not causation, but identity. It took me a long time (and many knocks about the head from smarter people) to realize that this line is directionless. If someone labels the top Meaningful and the bottom Meaningless, or the top Important and the bottom Unimportant, we see this at once for an error; but the same labels are still errors, if applied in the opposite order. If someone labels the top Contingent and the bottom Necessary, this is another error; if the top Subjective and the bottom Objective, another; or if the top Less Real and the bottom More Real, another still. (Some errors of this type have been called "reductionism," but they aren't the thing people mean when they say "reductionism" around here.) Whatever is, is real—and equally so, wherever it appears along the blue line.

At one time I would have labelled the top Emergent and the bottom Fundamental, but David Deutsch convinced me that even this was a mistake. Suppose we grant that a mind of arbitrary power, given only the bottom row, could deduce all the others—a popular hypothesis for which we have some good evidence (though not too much). Even then: could not this same mind, given only the complete row for Physiology, deduce the contents of Chemistry no less readily? The blue line has no direction; if I forget this I forget what identity means, and cast myself into confusion—the same type of confusion afflicting one who says, "Science believes that morality's not real!" Better, then, to unlabel the blue line entirely, and when someone wants to know what ontological difference exists between the higher rows and the lower, say "Mu." (Until I realized this I did not understand the metaethics sequence—but that isn't the topic of this post.)

Where does that leave reductionism? Right where it was, untouched. As finite entities, we never perceive the blue line as a whole—not a single azure band of the infinite expanse in which we live. We have to divide the line into graspable segments, and therefore must explain how each segment connects to the others; we must spin the green threads of explanation, drawing a map to overlay the territory. (The diagram is simplified here as well; a green thread is not as simple as "compounds explain synapse," but an intricate dance of analysis and synthesis.) In dividing the line we introduce relation between its divisions, and in introducing relation we introduce direction; emergence is a feature of maps, not of territories. (I would not say "The mind emerges from the brain," but "The active brain is the mind, and models of the mind emerge from models of the brain.") Reductionism proper is just this: noticing that green arrows are always present, and always point up. The whole is never more, nor less, than the sum of its parts; it only seems that way, if some parts have escaped our notice.



Now let's add a few more columns; again, we'll simplify the structure so we can see it, leaving out all worlds but one.



The violet lines are causation—how things are; together with the blue of what things are, they form indigo Reality: the World That Is. (Maybe the violet lines too have no direction; this sounds like timeless physics, of which I don't feel I can wisely speak.) In any case, they extend far beyond our reach, as each effect in turn becomes a cause; once again we find ourselves dividing the lines into portions we can grasp, and to restore the continuity we once removed, we spin threads of red. These too are explanations—though of another kind than the green. Just as violet lines connect blue lines to form a complete territory, red threads connect strands of green threads to form a complete map.



There are traps to fall into here, too. If we believed that the only violet lines (or the only red threads) that counted as real or meaningful were the ones on the bottom row, we would commit another error; this error also has been called "reductionism"—small wonder that it's sometimes deployed as a term of abuse! Or—because we are fallible and the true multiply-branching structure is hard to perceive—we might draw a red arrow pointing left, and be guilty of mere illogic.

But if we can braid red and green together, our best strength is here—in threads of gold. Only a golden thread is knowledge made whole, and no golden thread is ever spun but of green and red in harmony; until I know what a thing is and how it comes to be, both, I do not understand that thing.



If you're wondering about the empty space beneath, remember that the number of rows in the true structure is far above six; I suspect it is infinite. The number of columns is far higher than shown as well, so a golden thread connects, not a single point to a single point, but a wide expanse of one row to a wide expanse of the row above it. Golden threads are far-reaching theories and models—spun of many smaller explanations.

Often, those who find their cloth too threadbare for their taste will turn to another source of material: the beige threads of supernaturalism. Beige looks a bit like gold, if not examined too closely, and these threads have one great advantage: to spin them is the easiest thing in the world. Since they're unanchored to reality, you're free to craft them in any length or shape you like, lay them with arrows pointing wherever suits you, and even cover threads of red or green or gold whose lustre seems displeasing. Some have been taught to weave with beige alone, and in years of toil wrought patterns of strange and desolate beauty; but every hour of labour made their work, not more akin to fact, but less.



In spinning green and red, and in braiding them as gold, we become scientists; in cutting loose the snarls of beige, we become naturalists; in weaving our many threads into sturdy cloth, we become rationalists. Then we join our separate cloths as one, and in such tapestries—if all goes well—we glimpse truth: the harmony of indigo and gold.

Followup to: Building Phenomenological Bridges, Reductionism

Bridge hypotheses are extremely awkward. It's risky to draw permanent artificial lines between categories of hypothesis ('physical' vs. 'bridge'). We might not give the right complexity penalties to one kind of hypothesis relative to the other. Or we might implement a sensible framework for bridge hypotheses in one kind of brain that fails to predict the radically new phenomenology that results from expanding one's visual cortex onto new hardware.

We'd have to hope that it makes sense to talk about 'correct' bridging rules (correctly relating a hypothesis about external stimuli or about transistors composing yourself, to which settings are in fact the ones you call 'green'), even though they're quite different from ordinary physical descriptions of the world. And, since fully general and error-free knowledge of the phenomenologies of possible agents will probably not be available to a seed AGI or to its programmers, we'd have to hope that it's possible to build a self-modifying inductor robust enough that mistaken bridge predictions would just result in a quick Bayesian update towards better ideas. It's definitely a dangling thread.

Why, then, can't we do without them? Maybe they're a handy heuristic for agents with incomplete knowledge — but can they truly never be eliminated?

The notion of an irreducible divide between an AI's subjective sensations and its models of the objective world may sound suspiciously dualistic. If we live in a purely physical world, then why shouldn't a purely physical agent, once it’s come to a complete understanding of itself and the world, be able to dispense with explicit bridges? These are, after all, the agent's beliefs that we're talking about. In the limit, intuitively, accurate beliefs should just look like the world. So shouldn't the agent's phenomenological self-models eventually end up collapsing into its physical world-models — dispensing with a metaphysically basic self/world distinction?¹

Yes and no. When humans first began hypothesizing about the relationship between mind and matter, the former domain did not appear to be reducible to the latter. A number of philosophers concluded from this that there was a deep metaphysical divide between the two. But as the sciences of mind began to erode that belief in mind-matter dualism, they didn't eliminate the conceptual, linguistic, or intuitive distinctness of our mental and physical models. It may well be that we'll never abandon an intentional stance toward many phenomena, even once we've fully reduced them to their physical, biological, or computational underpinnings. Models of different levels can remain useful even once we've recognized that they co-refer.

In the case of an artificial scientist, beliefs in a fundamental sensation-v.-world dichotomy may dissolve even if the agent retains a useful conceptual distinction between its perceptual stream and the rest of the world. A lawful, unified physics need not be best modeled by agents with only a single world-modeling subprocess. 'There is one universe' doesn't imply 'one eye is optimal for viewing the universe'; 'there is one Earth' doesn't imply 'one leg is optimal for walking it'. The cases seem different chiefly because the leg/ground distinction is easier for humans to keep straight than the map/territory distinction.

Empirical reasoning requires a representational process that produces updates, and another representational process that gets updated. Eliminate the latter, and gone is the AI’s memory and expectation. (Imagine Cai experiencing its sequence of colors forever without considering any states of affairs they predict.) Eliminate the former, and the AGI has nothing but its frozen memories. (Imagine Cai without any sensory input, just a floating array of static world-models.) Keep both and eliminate bridging, and Cai painstakingly collects its visual data only to throw it all away; it has beliefs, but it never updates them.

Can we replace perceptions and expectations with a single kind-of-perceptiony kind-of-expectationish epistemic process, in a way that obviates any need for bridge hypotheses?

Maybe, but I don't know what that would look like. An agent's perceptions and its hypotheses are of different types, just by virtue of having distinct functions; and its meta-representations must portray them as such, lest its metacognitive reasoning fall into systemic error. Striving mightily to conflate the two may not make any more sense than striving to get an agent to smell colors or taste sounds.²

The only candidate I know of for a framework that may sidestep this distinction without thereby catching fire is Updateless Decision Theory, which was brought up by Jim Babcock, Vladimir Slepnev, and Wei Dei. UDT eliminates the need for bridge hypotheses in a particularly bold way, by doing away with updatable hypotheses altogether.

I don't understand UDT well enough to say how it bears on the problem of naturalizing induction, but I may return to this point when I have a better grasp on it. If UDT turns out to solve or dissolve the problem, it will be especially useful to have on hand a particular reductionism-related problem that afflicts other kinds of agents and is solved by UDT. This will be valuable even if UDT has other features that are undesirable enough to force us to come up with alternative solutions to naturalized induction.

For now, I'll just make a general point: It's usually good policy for an AGI to think like reality; but if an introspectible distinction between updatable information and update-causing information is useful for real-world inductors, then we shouldn't strip all traces of it from artificial reasoners, for much the same reason we shouldn't reduce our sensory apparatuses to a single modality in an attempt to ape the unity of our world's dynamics. Reductionism restricts what we can rationally believe about the territory, but it doesn't restrict the idiom of our maps.

¹ This is close to the worry Alex Flint raised, though our main concern is with the agent's ability to reduce its own mental types, since this is a less avoidable problem than a third party trying to do the same.

² The analogy to sensory modality is especially apt given that phenomenological bridge hypotheses can link sensory channels instead of linking a sensory channel to a hypothesized physical state. For instance, 'I see yellow whenever I taste isoamyl acetate' can function as a bridge between sensations an agent types as 'vision' and sensations an agent types as 'taste'.

Let's say Bob's terminal value is to travel back in time and ride a dinosaur.

It is instrumentally rational for Bob to study physics so he can learn how to build a time machine. As he learns more physics, Bob realizes that his terminal value is not only utterly impossible but meaningless. By definition, someone in Bob's past riding a dinosaur is not a future evolution of the present Bob.

There are a number of ways to create the subjective experience of having gone into the past and ridden a dinosaur. But to Bob, it's not the same because he wanted both the subjective experience and the knowledge that it corresponded to objective fact. Without the latter, he might as well have just watched a movie or played a video game.

So if we took the original, innocent-of-physics Bob and somehow calculated his coherent extrapolated volition, we would end up with a Bob who has given up on time travel. The original Bob would not want to be this Bob.

But, how do we know that _anything_ we value won't similarly dissolve under sufficiently thorough deconstruction? Let's suppose for a minute that all "human values" are dangling units; that everything we want is as possible and makes as much sense as wanting to hear the sound of blue or taste the flavor of a prime number. What is the rational course of action in such a situation?

PS: If your response resembles "keep attempting to XXX anyway", please explain what privileges XXX over any number of other alternatives other than your current preference. Are you using some kind of pre-commitment strategy to a subset of your current goals? Do you now wish you had used the same strategy to precommit to goals you had when you were a toddler?

Concretizing the abstract is an interesting blog post in that it makes a relatively cogent argument for non-reductionism. While I don't agree with it, I found it useful in that it helped me better understand how intelligent non-reductionists think. It also helped clarify to me an old distinction, that of philosophers versus engineers.

We abstract when we consider some particular aspect of a concrete thing while bracketing off or ignoring the other aspects of the thing.  For example, when you consider a dinner bell or the side of a pyramid exclusively as instances of triangularity, you ignore their color, size, function, and metal or stone composition.  Or to borrow an example from a recent post, when aircraft engineers determine how many passengers can be carried on a certain plane, they might focus exclusively on their average weight and ignore not only the passengers’ sex, ethnicity, hair color, dinner service preferences, etc., but even the actual weight of any particular passenger. [...]

Abstractions can be very useful, and are of themselves perfectly innocent when we keep in mind that we are abstracting.  The trouble comes when we start to think of abstractions as if they were concrete realities themselves -- thereby “reifying” them -- and especially when we think of the abstractions as somehow more real than the concrete realities from which they have been abstracted. [...]

I do not mean to deny that abstractions of the sort in question may have their uses.  On the contrary, the mathematical conception of matter is extremely useful, as the astounding technologies that surround us in modern life make obvious.  But contrary to what some proponents of scientism suppose, it simply doesn’t follow for a moment that that conception gives us an exhaustive conception of the material world, for reasons I have stated many times (e.g. here). [...]

Then there is social science.  When we abstract from concrete human beings their purely economic motivations, ignoring everything else and then reifying this abstraction, the result is homo economicus, a strange creature who, unlike real people, is driven by nothing but the desire to maximize utility.  Nietzschean analyses of human motivation in terms of the will to power are less susceptible of mathematical modeling (and thus less “scientific”), but are variations on the same sort of error.  Evolutionary psychology often combines abstractions of the natural scientific and social scientific sort.  Like the neuroscientist, the evolutionary psychologist often treats parts of human beings as if they were substances independent of the whole from which they have been abstracted (”selfish genes,” “memes”), and adds to this reification the abstractions of the economist (e.g. game theory).

As the neuroscientific and sociobiological examples indicate, the Reification Fallacy is often combined with other fallacies.  In these cases, parts of a whole substance are first abstracted from it and treated as if they were substances in their own right (e.g. brain hemispheres, genes); and then a second, “Mereological Fallacy” (as Bennett and Hacker call it) is committed, in which what is intelligibly attributed only to the whole is attributed to the parts (e.g. the left hemisphere of the brain is said to “interpret,” and genes are said to be “selfish”). [...]

The irony is that while New Atheists and others beholden to scientism pride themselves on being “reality based,” that is precisely what they are not.  Actual, concrete reality is extremely complicated.  There is far more to material systems than what can be captured in the equations of physics, far more to human beings than can be captured in the categories of neuroscience or economics, and far more to religion than can be captured in the ludicrous straw men peddled by New Atheists.  All of these simplifying abstractions (except the last) have their value, but when we treat them as anything more than simplifying abstractions we have left the realm of science and entered that of ideology.  The varieties of reductionism, eliminativism, and the “hermeneutics of suspicion” are manifestations of this tendency to replace real things with abstractions.  They are all attempts to “conquer the abundance” of reality (as Paul Feyerabend might have put it), to force the world in all its concrete richness into a straightjacket.

I find this interesting in the way that smart people are likely to disagree with the correct interpretation of some of its claims - while others would say the post is worshipping the mysterious, others would say that it's just making reasonable cautions about the inherent methodological limitations of a certain approach. One might even think that it's essentially making a similar point as Eliezer's warning about floating beliefs, and therefore to agree with the Sequences. The caution of "beware of thinking that your abstractions say everything that there is to be said about something" is a reasonable one, and people do clearly make that mistake sometimes.

I expect that part of what influences how plausible one finds this argument depends on whether one has more of an "engineer's mindset" or a "philosopher's mindset". Somebody with an engineer's mindset will think that "yes, the abstractions we use might be imperfect, but what else do you propose we use? They're still the best tool for accomplishing stuff, and anything else is just philosophcial nonsense that isn't grounded in anything". Whereas the philosopher is less interested in using their knowledge to "accomplish stuff", and more interested in the ideas and their implications themselves.

As an aside, this distinction might be part of the reason why we have so many computer or hard science folks on this site. Partially it's because Eliezer used a lot of CS jargon in writing the Sequences, but probably also because the Sequences, while philosophical in nature, are also very focused on practical results and getting empirical predictions out of your beliefs.

Looking at what we could use this distinction for (and thus taking an engineer's mindset) some people here have mentioned getting an "ick" reaction from religious people, just due to those people having strong false beliefs. I think that, combined with properly understanding the emotional basis of religion, an understanding of the philosopher / engineer distinction can help avoid that reaction. Our values determine our beliefs, and there are plenty of religious people who aren't stupid, crazy, or anything like that. They might simply be philosophers instead of engineers, or they might be engineers who are more interested in the instrumental benefits of religion than the rather marginal benefits of x-rationality. (Amusingly, such a "religious engineer" might justifiably consider our obsession with "truth" as just an odd philosophical pursuit.)

The poem is from someone whose online pseudonym is atiguhya padma.  I'll quote the first verse, the refrain, and the beginning of the second verse to give you enough flavor to decide if you want to follow the link.  There are about 9 verses total.

Previous articles: Personal research update, Does functionalism imply dualism?, State your physical account of experienced color.

In phenomenology, there is a name for the world of experience, the "lifeworld". The lifeworld is the place where you exist, where time flows, and where things are actually green. One of the themes of the later work of Edmund Husserl is that a scientific image of the real world has been constructed, on the basis of which it is denied that various phenomena of the lifeworld exist anywhere, at any level of reality.

When I asked, in the previous post, for a few opinions about what color is and how it relates to the world according to current science, I was trying to gauge just how bad the eclipse of the lifeworld by theoretical conceptions is, among the readers of this site. I'd say there is a problem, but it's a problem that might be solved by patient discussion.

Someone called Automaton has given us a clear statement of the extreme position: nothing is actually green at any level of reality; even green experiences don't involve the existence of anything that is actually green; there is no green in reality, there is only "experience of green" which is not itself green. I see other responses which are just a step or two away from this extreme, but they don't deny the existence of actual color with that degree of unambiguity.

A few people talk about wavelengths of light, but I doubt that they want to assert that the light in question, as it traverses space, is actually colored green. Which returns us to the dilemma: either "experiences" exist and part of them is actually green, or you have to say that nothing exists, in any sense, at any level of reality, that is actually green. Either the lifeworld exists somewhere in reality, or you must assert, as does the philosopher quoted by Automaton, that all that exists are brain processes and words. Your color sensations aren't really there, you're "having a sensation" without there being a sensation in reality.

What about the other responses? kilobug seems to think that pi actually exists inside a computer calculating the digits of pi, and that this isn't dualist. Manfred thinks that "keeping definitions and referents distinct" would somehow answer the question of where in reality the actual shades of green are. drethelin says "The universe does not work how it feels to us it works" without explaining in physical terms what these feelings about reality are, and whether any of them is actually green. pedanterrific asks why wrangle about color rather than some other property (the answer is that the case of color makes this sort of problem as obvious as it ever gets). RomeoStevens suggests I look into Jeff Hawkins. Hawkins mentions qualia once in his book "On Intelligence", where he speculates about what sort of neural encoding might be the physical correlate of a color experience; but he doesn't say how or whether anything manages to be actually colored.

amcknight asks which of 9 theories of color listed in the SEP article on that subject I'm talking about. If you go a few paragraphs back from the list of 9 theories, you will see references to "color as it is in experience" or "color as a subjective quality". That's the type of color I'm talking about. The 9 theories are all ways of talking about "color as in physical objects", and focus on the properties of the external stimuli which cause a color sensation. The article gets around to talking about actual color, subjective or "phenomenal" color, only at the end.

Richard Kennaway comes closest to my position; he calls it an apparently impossible situation which we are actually living. I wouldn't put it quite like that; the only reason to call it impossible is if you are completely invested in an ontology lacking the so-called secondary qualities; if you aren't, it's just a problem to solve, not a paradox. But Richard comes closest (though who knows what Will Newsome is thinking). LW user "scientism" bites a different bullet to the eliminativists, and says colors are real and are properties of the external objects. That gets a point for realism, but it doesn't explain color in a dream or a hallucination.

Changing people's minds on this subject is an uphill battle, but people here are willing to talk, and most of these subjects have already been discussed for decades. There's ample opportunity to dissolve, not the problem, but the false solutions which only obscure the real problem, by drawing on the work of others; preferably before the future Rationality Institute starts mass-producing people who have the vice of quale-blindness as well as the virtues of rationality. Some of those people will go on to work on Friendly AI. So it's highly desirable that someone should do this. However, that would require time that I no longer have.

In this series of posts, I certainly didn't set out to focus on the issue of color. The first post is all about Friendly AI, the ontology of consciousness, and a hypothetical future discipline of quantum neurobiology. It may still be unclear why I think evidence for quantum computing in the brain could help with the ontological problems of consciousness. I feel that the brief discussion this week has produced some minor progress in explaining myself, which needs to be consolidated into something better. But see my remarks here about being able to collapse the dualistic distinction between mental and physical ontology in a tensor network ontology; also earlier remarks here about about mathematically representing the phenomenological ontology of consciousness. I don't consider myself dogmatic about what the answer is, just about the inadequacy of all existing solutions, though I respect my own ideas enough to want to pursue them, and to believe that doing so will be usefully instructive, even if they are wrong.

However, my time is up. In real life, my ability to continue even at this inadequate level hangs by a thread. I don't mean that I'm suicidal, I mean that I can't eat air. I spent a year getting to this level in physics, so I could perform this task. I have considerable momentum now, but it will go to waste unless I can keep going for a little longer - a few weeks, maybe a few months. That should be enough time to write something up that contains a result of genuine substance, and/or enough time to secure an economic basis for my existence in real life that permits me to keep going. I won't go into detail here about how slim my resources really are, or how adverse my conditions, but it has been the effort that you would want from someone who has important contributions to make, and nowhere to turn for direct assistance.[*] I've done what I can, these posts are the end of it, and the next few days will decide whether I can keep going, or whether I have to shut down my brain once again.

So, one final remark. Asking for donations doesn't seem to work yet. So what if I promise to pay you back? Then the only cost you bear is the opportunity cost and the slight risk of default. Ten years ago, Eliezer lent me the airfare to Atlanta for a few days of brainstorming. It took a while, but he did get that money back. I honor my commitments and this one is highly public. This really is the biggest bargain in existential risk mitigation and conceptual boundary-breaking that you'll ever get: not even a gift, just a loan is required. If you want to discuss a deal, don't do it here, but mail me at mitchtemporarily@hotmail.com. One person might be enough to make the difference.

[*]Really, I can't say that, that's an emotional statement. There has been lots of assistance, large and small, from people in my life. But it's been a struggle conducted at subsistence level the whole way.

ETA 6 Feb: I get to keep going.

Synopsis: The brain is a quantum computer and the self is a tensor factor in it - or at least, the truth lies more in that direction than in the classical direction - and we won't get Friendly AI right unless we get the ontology of consciousness right.

Followed by: Does functionalism imply dualism?

Sixteen months ago, I made a post seeking funding for personal research. There was no separate Discussion forum then, and the post was comprehensively downvoted. I did manage to keep going at it, full-time, for the next sixteen months. Perhaps I'll get to continue; it's for the sake of that possibility that I'll risk another breach of etiquette. You never know who's reading these words and what resources they have. Also, there has been progress.

I think the best place to start is with what orthonormal said in response to the original post: "I don't think anyone should be funding a Penrose-esque qualia mysterian to study string theory." If I now took my full agenda to someone out in the real world, they might say: "I don't think it's worth funding a study of 'the ontological problem of consciousness in the context of Friendly AI'." That's my dilemma. The pure scientists who might be interested in basic conceptual progress are not engaged with the race towards technological singularity, and the apocalyptic AI activists gathered in this place are trying to fit consciousness into an ontology that doesn't have room for it. In the end, if I have to choose between working on conventional topics in Friendly AI, and on the ontology of quantum mind theories, then I have to choose the latter, because we need to get the ontology of consciousness right, and it's possible that a breakthrough could occur in the world outside the FAI-aware subculture and filter through; but as things stand, the truth about consciousness would never be discovered by employing the methods and assumptions that prevail inside the FAI subculture.

Perhaps I should pause to spell out why the nature of consciousness matters for Friendly AI. The reason is that the value system of a Friendly AI must make reference to certain states of conscious beings - e.g. "pain is bad" - so, in order to make correct judgments in real life, at a minimum it must be able to tell which entities are people and which are not. Is an AI a person? Is a digital copy of a human person, itself a person? Is a human body with a completely prosthetic brain still a person?

I see two ways in which people concerned with FAI hope to answer such questions. One is simply to arrive at the right computational, functionalist definition of personhood. That is, we assume the paradigm according to which the mind is a computational state machine inhabiting the brain, with states that are coarse-grainings (equivalence classes) of exact microphysical states. Another physical system which admits the same coarse-graining - which embodies the same state machine at some macroscopic level, even though the microscopic details of its causality are different - is said to embody another instance of the same mind.

An example of the other way to approach this question is the idea of simulating a group of consciousness theorists for 500 subjective years, until they arrive at a consensus on the nature of consciousness. I think it's rather unlikely that anyone will ever get to solve FAI-relevant problems in that way. The level of software and hardware power implied by the capacity to do reliable whole-brain simulations means you're already on the threshold of singularity: if you can simulate whole brains, you can simulate part brains, and you can also modify the parts, optimize them with genetic algorithms, and put them together into nonhuman AI. Uploads won't come first.

But the idea of explaining consciousness this way, by simulating Daniel Dennett and David Chalmers until they agree, is just a cartoon version of similar but more subtle methods. What these methods have in common is that they propose to outsource the problem to a computational process using input from cognitive neuroscience. Simulating a whole human being and asking it questions is an extreme example of this (the simulation is the "computational process", and the brain scan it uses as a model is the "input from cognitive neuroscience"). A more subtle method is to have your baby AI act as an artificial neuroscientist, use its streamlined general-purpose problem-solving algorithms to make a causal model of a generic human brain, and then to somehow extract from that, the criteria which the human brain uses to identify the correct scope of the concept "person". It's similar to the idea of extrapolated volition, except that we're just extrapolating concepts.

It might sound a lot simpler to just get human neuroscientists to solve these questions. Humans may be individually unreliable, but they have lots of cognitive tricks - heuristics - and they are capable of agreeing that something is verifiably true, once one of them does stumble on the truth. The main reason one would even consider the extra complication involved in figuring out how to turn a general-purpose seed AI into an artificial neuroscientist, capable of extracting the essence of the human decision-making cognitive architecture and then reflectively idealizing it according to its own inherent criteria, is shortage of time: one wishes to develop friendly AI before someone else inadvertently develops unfriendly AI. If we stumble into a situation where a powerful self-enhancing algorithm with arbitrary utility function has been discovered, it would be desirable to have, ready to go, a schema for the discovery of a friendly utility function via such computational outsourcing.

Now, jumping ahead to a later stage of the argument, I argue that it is extremely likely that distinctively quantum processes play a fundamental role in conscious cognition, because the model of thought as distributed classical computation actually leads to an outlandish sort of dualism. If we don't concern ourselves with the merits of my argument for the moment, and just ask whether an AI neuroscientist might somehow overlook the existence of this alleged secret ingredient of the mind, in the course of its studies, I do think it's possible. The obvious noninvasive way to form state-machine models of human brains is to repeatedly scan them at maximum resolution using fMRI, and to form state-machine models of the individual voxels on the basis of this data, and then to couple these voxel-models to produce a state-machine model of the whole brain. This is a modeling protocol which assumes that everything which matters is physically localized at the voxel scale or smaller. Essentially we are asking, is it possible to mistake a quantum computer for a classical computer by performing this sort of analysis? The answer is definitely yes if the analytic process intrinsically assumes that the object under study is a classical computer. If I try to fit a set of points with a line, there will always be a line of best fit, even if the fit is absolutely terrible. So yes, one really can describe a protocol for AI neuroscience which would be unable to discover that the brain is quantum in its workings, and which would even produce a specific classical model on the basis of which it could then attempt conceptual and volitional extrapolation.

Clearly you can try to circumvent comparably wrong outcomes, by adding reality checks and second opinions to your protocol for FAI development. At a more down to earth level, these exact mistakes could also be made by human neuroscientists, for the exact same reasons, so it's not as if we're talking about flaws peculiar to a hypothetical "automated neuroscientist". But I don't want to go on about this forever. I think I've made the point that wrong assumptions and lax verification can lead to FAI failure. The example of mistaking a quantum computer for a classical computer may even have a neat illustrative value. But is it plausible that the brain is actually quantum in any significant way? Even more incredibly, is there really a valid apriori argument against functionalism regarding consciousness - the identification of consciousness with a class of computational process?

I have previously posted (here) about the way that an abstracted conception of reality, coming from scientific theory, can motivate denial that some basic appearance corresponds to reality. A perennial example is time. I hope we all agree that there is such a thing as the appearance of time, the appearance of change, the appearance of time flowing... But on this very site, there are many people who believe that reality is actually timeless, and that all these appearances are only appearances; that reality is fundamentally static, but that some of its fixed moments contain an illusion of dynamism.

The case against functionalism with respect to conscious states is a little more subtle, because it's not being said that consciousness is an illusion; it's just being said that consciousness is some sort of property of computational states. I argue first that this requires dualism, at least with our current physical ontology, because conscious states are replete with constituents not present in physical ontology - for example, the "qualia", an exotic name for very straightforward realities like: the shade of green appearing in the banner of this site, the feeling of the wind on your skin, really every sensation or feeling you ever had. In a world made solely of quantum fields in space, there are no such things; there are just particles and arrangements of particles. The truth of this ought to be especially clear for color, but it applies equally to everything else.

In order that this post should not be overlong, I will not argue at length here for the proposition that functionalism implies dualism, but shall proceed to the second stage of the argument, which does not seem to have appeared even in the philosophy literature. If we are going to suppose that minds and their states correspond solely to combinations of mesoscopic information-processing events like chemical and electrical signals in the brain, then there must be a mapping from possible exact microphysical states of the brain, to the corresponding mental states. Supposing we have a mapping from mental states to coarse-grained computational states, we now need a further mapping from computational states to exact microphysical states. There will of course be borderline cases. Functional states are identified by their causal roles, and there will be microphysical states which do not stably and reliably produce one output behavior or the other.

Physicists are used to talking about thermodynamic quantities like pressure and temperature as if they have an independent reality, but objectively they are just nicely behaved averages. The fundamental reality consists of innumerable particles bouncing off each other; one does not need, and one has no evidence for, the existence of a separate entity, "pressure", which exists in parallel to the detailed microphysical reality. The idea is somewhat absurd.

Yet this is analogous to the picture implied by a computational philosophy of mind (such as functionalism) applied to an atomistic physical ontology. We do know that the entities which constitute consciousness - the perceptions, thoughts, memories... which make up an experience - actually exist, and I claim it is also clear that they do not exist in any standard physical ontology. So, unless we get a very different physical ontology, we must resort to dualism. The mental entities become, inescapably, a new category of beings, distinct from those in physics, but systematically correlated with them. Except that, if they are being correlated with coarse-grained neurocomputational states which do not have an exact microphysical definition, only a functional definition, then the mental part of the new combined ontology is fatally vague. It is impossible for fundamental reality to be objectively vague; vagueness is a property of a concept or a definition, a sign that it is incomplete or that it does not need to be exact. But reality itself is necessarily exact - it is something - and so functionalist dualism cannot be true unless the underdetermination of the psychophysical correspondence is replaced by something which says for all possible physical states, exactly what mental states (if any) should also exist. And that inherently runs against the functionalist approach to mind.

Very few people consider themselves functionalists and dualists. Most functionalists think of themselves as materialists, and materialism is a monism. What I have argued is that functionalism, the existence of consciousness, and the existence of microphysical details as the fundamental physical reality, together imply a peculiar form of dualism in which microphysical states which are borderline cases with respect to functional roles must all nonetheless be assigned to precisely one computational state or the other, even if no principle tells you how to perform such an assignment. The dualist will have to suppose that an exact but arbitrary border exists in state space, between the equivalence classes.

This - not just dualism, but a dualism that is necessarily arbitrary in its fine details - is too much for me. If you want to go all Occam-Kolmogorov-Solomonoff about it, you can say that the information needed to specify those boundaries in state space is so great as to render this whole class of theories of consciousness not worth considering. Fortunately there is an alternative.

Here, in addressing this audience, I may need to undo a little of what you may think you know about quantum mechanics. Of course, the local preference is for the Many Worlds interpretation, and we've had that discussion many times. One reason Many Worlds has a grip on the imagination is that it looks easy to imagine. Back when there was just one world, we thought of it as particles arranged in space; now we have many worlds, dizzying in their number and diversity, but each individual world still consists of just particles arranged in space. I'm sure that's how many people think of it.

Among physicists it will be different. Physicists will have some idea of what a wavefunction is, what an operator algebra of observables is, they may even know about path integrals and the various arcane constructions employed in quantum field theory. Possibly they will understand that the Copenhagen interpretation is not about consciousness collapsing an actually existing wavefunction; it is a positivistic rationale for focusing only on measurements and not worrying about what happens in between. And perhaps we can all agree that this is inadequate, as a final description of reality. What I want to say, is that Many Worlds serves the same purpose in many physicists' minds, but is equally inadequate, though from the opposite direction. Copenhagen says the observables are real but goes misty about unmeasured reality. Many Worlds says the wavefunction is real, but goes misty about exactly how it connects to observed reality. My most frustrating discussions on this topic are with physicists who are happy to be vague about what a "world" is. It's really not so different to Copenhagen positivism, except that where Copenhagen says "we only ever see measurements, what's the problem?", Many Worlds says "I say there's an independent reality, what else is left to do?". It is very rare for a Many World theorist to seek an exact idea of what a world is, as you see Robin Hanson and maybe Eliezer Yudkowsky doing; in that regard, reading the Sequences on this site will give you an unrepresentative idea of the interpretation's status.

One of the characteristic features of quantum mechanics is entanglement. But both Copenhagen, and a Many Worlds which ontologically privileges the position basis (arrangements of particles in space), still have atomistic ontologies of the sort which will produce the "arbitrary dualism" I just described. Why not seek a quantum ontology in which there are complex natural unities - fundamental objects which aren't simple - in the form of what we would presently called entangled states? That was the motivation for the quantum monadology described in my other really unpopular post. :-) [Edit: Go there for a discussion of "the mind as tensor factor", mentioned at the start of this post.] Instead of saying that physical reality is a series of transitions from one arrangement of particles to the next, say it's a series of transitions from one set of entangled states to the next. Quantum mechanics does not tell us which basis, if any, is ontologically preferred. Reality as a series of transitions between overall wavefunctions which are partly factorized and partly still entangled is a possible ontology; hopefully readers who really are quantum physicists will get the gist of what I'm talking about.

I'm going to double back here and revisit the topic of how the world seems to look. Hopefully we agree, not just that there is an appearance of time flowing, but also an appearance of a self. Here I want to argue just for the bare minimum - that a moment's conscious experience consists of a set of things, events, situations... which are simultaneously "present to" or "in the awareness of" something - a conscious being - you. I'll argue for this because even this bare minimum is not acknowledged by existing materialist attempts to explain consciousness. I was recently directed to this brief talk about the idea that there's no "real you". We are given a picture of a graph whose nodes are memories, dispositions, etc., and we are told that the self is like that graph: nodes can be added, nodes can be removed, it's a purely relational composite without any persistent part. What's missing in that description is that bare minimum notion of a perceiving self. Conscious experience consists of a subject perceiving objects in certain aspects. Philosophers have discussed for centuries how best to characterize the details of this phenomenological ontology; I think the best was Edmund Husserl, and I expect his work to be extremely important in interpreting consciousness in terms of a new physical ontology. But if you can't even notice that there's an observer there, observing all those parts, then you won't get very far.

My favorite slogan for this is due to the other Jaynes, Julian Jaynes. I don't endorse his theory of consciousness at all; but while in a daydream he once said to himself, "Include the knower in the known". That sums it up perfectly. We know there is a "knower", an experiencing subject. We know this, just as well as we know that reality exists and that time passes. The adoption of ontologies in which these aspects of reality are regarded as unreal, as appearances as only, may be motivated by science, but it's false to the most basic facts there are, and one should show a little more imagination about what science will say when it's more advanced.

I think I've said almost all of this before. The high point of the argument is that we should look for a physical ontology in which a self exists and is a natural yet complex unity, rather than a vaguely bounded conglomerate of distinct information-processing events, because the latter leads to one of those unacceptably arbitrary dualisms. If we can find a physical ontology in which the conscious self can be identified directly with a class of object posited by the theory, we can even get away from dualism, because physical theories are mathematical and formal and make few commitments about the "inherent qualities" of things, just about their causal interactions. If we can find a physical object which is absolutely isomorphic to a conscious self, then we can turn the isomorphism into an identity, and the dualism goes away. We can't do that with a functionalist theory of consciousness, because it's a many-to-one mapping between physical and mental, not an isomorphism.

So, I've said it all before; what's new? What have I accomplished during these last sixteen months? Mostly, I learned a lot of physics. I did not originally intend to get into the details of particle physics - I thought I'd just study the ontology of, say, string theory, and then use that to think about the problem. But one thing led to another, and in particular I made progress by taking ideas that were slightly on the fringe, and trying to embed them within an orthodox framework. It was a great way to learn, and some of those fringe ideas may even turn out to be correct. It's now abundantly clear to me that I really could become a career physicist, working specifically on fundamental theory. I might even have to do that, it may be the best option for a day job. But what it means for the investigations detailed in this essay, is that I don't need to skip over any details of the fundamental physics. I'll be concerned with many-body interactions of biopolymer electrons in vivo, not particles in a collider, but an electron is still an electron, an elementary particle, and if I hope to identify the conscious state of the quantum self with certain special states from a many-electron Hilbert space, I should want to understand that Hilbert space in the deepest way available.

My only peer-reviewed publication, from many years ago, picked out pathways in the microtubule which, we speculated, might be suitable for mobile electrons. I had nothing to do with noticing those pathways; my contribution was the speculation about what sort of physical processes such pathways might underpin. Something I did notice, but never wrote about, was the unusual similarity (so I thought) between the microtubule's structure, and a model of quantum computation due to the topologist Michael Freedman: a hexagonal lattice of qubits, in which entanglement is protected against decoherence by being encoded in topological degrees of freedom. It seems clear that performing an ontological analysis of a topologically protected coherent quantum system, in the context of some comprehensive ontology ("interpretation") of quantum mechanics, is a good idea. I'm not claiming to know, by the way, that the microtubule is the locus of quantum consciousness; there are a number of possibilities; but the microtubule has been studied for many years now and there's a big literature of models... a few of which might even have biophysical plausibility.

As for the interpretation of quantum mechanics itself, these developments are highly technical, but revolutionary. A well-known, well-studied quantum field theory turns out to have a bizarre new nonlocal formulation in which collections of particles seem to be replaced by polytopes in twistor space. Methods pioneered via purely mathematical studies of this theory are already being used for real-world calculations in QCD (the theory of quarks and gluons), and I expect this new ontology of "reality as a complex of twistor polytopes" to carry across as well. I don't know which quantum interpretation will win the battle now, but this is new information, of utterly fundamental significance. It is precisely the sort of altered holistic viewpoint that I was groping towards when I spoke about quantum monads constituted by entanglement. So I think things are looking good, just on the pure physics side. The real job remains to show that there's such a thing as quantum neurobiology, and to connect it to something like Husserlian transcendental phenomenology of the self via the new quantum formalism.

It's when we reach a level of understanding like that, that we will truly be ready to tackle the relationship between consciousness and the new world of intelligent autonomous computation. I don't deny the enormous helpfulness of the computational perspective in understanding unconscious "thought" and information processing. And even conscious states are still states, so you can surely make a state-machine model of the causality of a conscious being. It's just that the reality of how consciousness, computation, and fundamental ontology are connected, is bound to be a whole lot deeper than just a stack of virtual machines in the brain. We will have to fight our way to a new perspective which subsumes and transcends the computational picture of reality as a set of causally coupled black-box state machines. It should still be possible to "port" most of the thinking about Friendly AI to this new ontology; but the differences, what's new, are liable to be crucial to success. Fortunately, it seems that new perspectives are still possible; we haven't reached Kantian cognitive closure, with no more ontological progress open to us. On the contrary, there are still lines of investigation that we've hardly begun to follow.

A great column by Chris Sims at the Comics Alliance.

Excerpt:

Because that's the thing about Scooby-Doo: The bad guys in every episode aren't monsters, they're liars.

I can't imagine how scandalized those critics who were relieved to have something that was mild enough to not excite their kids would've been if they'd stopped for a second and realized what was actually going on. The very first rule of Scooby-Doo, the single premise that sits at the heart of their adventures, is that the world is full of grown-ups who lie to kids, and that it's up to those kids to figure out what those lies are and call them on it, even if there are other adults who believe those lies with every fiber of their being. And the way that you win isn't through supernatural powers, or even through fighting. The way that you win is by doing the most dangerous thing that any person being lied to by someone in power can do: You think.

Tim Minchin fans may recall him mentioning Scooby Doo in a similar light in his beat poem Storm, and it's been brought up on Less Wrong before.

When viewed in this light, Scooby Doo really is like an elementary version of Methods of Rationality.

For your consideration, a psychology study as summarized by The Economist in "How dead is dead? Sometimes, those who have died seem more alive than those who have not":

"They first asked 201 people stopped in public in New York and New England to answer questions after reading one of three short stories. In all three, a man called David was involved in a car accident and suffered serious injuries. In one, he recovered fully. In another, he died. In the third, his entire brain was destroyed except for one part that kept him breathing. Although he was technically alive, he would never again wake up.

...each participant was asked to rate David’s mental capacities, including whether he could influence the outcome of events, know right from wrong, remember incidents from his life, be aware of his environment, possess a personality and have emotions. Participants used a seven-point scale to make these ratings, where 3 indicated that they strongly agreed that he could do such things...and -3 indicated that they strongly disagreed.

...the fully recovered David rated an average of +1.77 and the dead David -0.29. That score for the dead David was surprising enough, suggesting as it did a considerable amount of mental acuity in the dead. What was extraordinary, though, was the result for the vegetative David: -1.73. In the view of the average New Yorker or New Englander, the vegetative David was more dead [-1.73] than the version who was dead [-0.29].

...they ran a follow-up experiment which had two different descriptions of the dead David. One said he had simply passed away. The other directed the participant’s attention to the corpse. It read, “After being embalmed at the morgue, he was buried in the local cemetery. David now lies in a coffin underground.”...In this follow-up study participants were also asked to rate how religious they were.

Once again, the vegetative David was seen to have less mind than the David who had “passed away”. This was equally true, regardless of how religious a participant said he was. However, ratings of the dead David’s mind in the story in which his corpse was embalmed and buried varied with the participant’s religiosity. Irreligious participants gave the buried corpse about the same mental ratings as the vegetative patient (-1.51 and -1.64 respectively). Religious participants, however, continued to ascribe less mind to the irretrievably unconscious David than they did to his buried corpse (-1.57 and 0.59).

That those who believe in an afterlife ascribe mental acuity to the dead is hardly surprising. That those who do not are inclined to do so unless heavily prompted not to is curious indeed."

The study is "More dead than dead: Perceptions of persons in the persistent vegetative state":

Patients in persistent vegetative state (PVS) may be biologically alive, but these experiments indicate that people see PVS as a state curiously more dead than dead. Experiment 1 found that PVS patients were perceived to have less mental capacity than the dead. Experiment 2 explained this effect as an outgrowth of afterlife beliefs, and the tendency to focus on the bodies of PVS patients at the expense of their minds. Experiment 3 found that PVS is also perceived as “worse” than death: people deem early death better than being in PVS. These studies suggest that people perceive the minds of PVS patients as less valuable than those of the dead – ironically, this effect is especially robust for those high in religiosity.

Ed Yong points to another interesting study, the 2004 "The natural emergence of reasoning about the afterlife as a developmental regularity":

Participants were interviewed about the biological and psychological functioning of a dead agent. In Experiment 1, even 4- to 6-year-olds stated that biological processes ceased at death, although this trend was more apparent among 6- to 8-year-olds. In Experiment 2, 4- to 12-year-olds were asked about psychological functioning. The youngest children were equally likely to state that both cognitive and psychobiological states continued at death, whereas the oldest children were more likely to state that cognitive states continued. In Experiment 3, children and adults were asked about an array of psychological states. With the exception of preschoolers, who did not differentiate most of the psychological states, older children and adults were likely to attribute epistemic, emotional, and desire states to dead agents. These findings suggest that developmental mechanisms underlie intuitive accounts of dead agents' minds

Jach on Hacker News makes the obvious connection with cryonics; see also lukeprog's "Remind Physicalists They're Physicalists".

I've recently read through Eliezer's sequence on "free will", and I generally found it to be a fairly satisfying resolution/dissolution of the many misunderstandings involved in standard debates about the subject. There's no conflict between saying "your past circumstances determined that you would rush into the burning orphanage" and "you decided to rush into the burning orphanage"; what really matters is the experience of weighing possible options against your emotions and morals, without knowledge of what you will decide, rather than some hypothetical freedom to have done something different, etc. Basically, the experience of deciding between alternatives is real, don't worry too much about nonsense philosophical "free will" debates, just move on and live your life. Fine.

But I'm trying to figure out the best way to conceptualize the idea that certain biological conditions can "inhibit" your "free will," even under a reductionist understanding of the concept. Consider this recent article in The Atlantic called "The Brain on Trial." The basic argument is that we have much less control over ourselves than we think, that biology and upbringing have tremendous influences on our decisions, and that the criminal justice system needs to account for the pervasiveness of biological influence on our actions. On the one hand, duh. The article treats the idea that we are "just" our biology as some kind of big revelation that has only recently been understood:

The crux of the problem is that it no longer makes sense to ask, “To what extent was it his biology, and to what extent was it him?,” because we now understand that there is no meaningful distinction between a person’s biology and his decision-making. They are inseparable.

Is that because we've just now discovered reductionism? If we weren't "just" our biology, what would we be? Magic? Whatever we mean by consciousness and decision-making, I'm sure LW members pretty much all accept that they occur within physics. The author doesn't even seem to fully grasp this point himself, because he states at the end that there "may" be at least some space for free will, independent of our biology, but that we just don't understand it yet:

Free will may exist (it may simply be beyond our current science), but one thing seems clear: if free will does exist, it has little room in which to operate. It can at best be a small factor riding on top of vast neural networks shaped by genes and environment.

Obviously most LW reductionists are going to immediately grasp that "free will" doesn't exist in addition to our neural networks. What would that even mean? It's not "90% neural networks, 10% free will" -- the point is that the process of your neural networks operating normally on a particular decision is what we mean by "free will," at least when we care to use that concept. (If anyone thinks I've stated this incorrectly, feel free to correct me.)

But still, notwithstanding that a lot of this article sort of seems to be missing the point (largely because the author doesn't quite get how obvious the central premise really is), I'm still wrestling with how to understand some of its more specific points, within the reductionist understanding of free will. For example, Charles Whitman, the shooter who killed 13 people from the UT Tower, had written out a suicide note noting that he had recently been the "victim of many unusual and irrational thoughts" and requesting that his brain be examined. An autopsy revealed that he had a large brain tumor that had damaged his amygdala, thus causing emotional and social disturbances. Similarly, in 2000, a man named "Alex" (fake name, but real case) suddenly developed pedophilic impulses at age 40, and was eventually convicted of child molestation. Turns out he also had a brain tumor, and once it was removed, his sexual interests went back to normal. The pedophilic impulses soon returned, and the doctors discovered the tumor had grown back -- they removed it for good, and his behavior went back to normal.

Obviously people like Charles and Alex aren't "victims of their biology" anymore than the rest of us. Nobody's brain has some magic "free will" space that "exempts" the person from biology. But even under the reductionist conception of free will, it still seems like Charles and Alex are somehow "less free" than "normal" people. Even though everyone's decisions are, in some sense, determined by their past circumstances, there still seems to be a meaningful way in which Charles are Alex are less able to make decisions "for themselves" than those of us without brain tumors -- almost as if they had a tick which caused involuntary physical actions, but drawn out over time in patterns, rather than in single bursts. Or to put it differently, where the phrase "your past circumstances determine who you are when you face a choice, you are still the one that decides" holds true for most people, it seems like it doesn't hold true for them. At the very least, it seems like we would certainly be justified in judging Charles and Alex differently from people who don't suffer from brain tumors.

But if we're already committed to the reductionist understanding of free will in the first place, what does this intuition that Charles and Alex are somehow "less free" really mean? Obviously we all have biological impulses that make us more or less inclined to make certain decisions, and that might therefore impede on some ideal conception of "control" over ourselves. But are these impulses qualitatively different from biological conditions that "override" normal decision-making? Is a brain tumor pushing on your amygdala more akin to prison bars that really do inhibit your free will in a purely physical sense, or just a more intense version of genes that give you a slight disposition toward violent behavior?

My intuition is that somewhere along the line here I may be asking a "wrong question," or importing some remnant of a non-biological conception of free will into my thinking. But I can't quite pin this issue down in a way that really resolves the answer in a satisfying way, so I was hoping that some of you might be able to help me reason through this appropriately. Thoughts?

Background

I was raised in the Churches of Christ and my family is all very serious about Christianity. About 3 years ago, I started to ask some hard questions, and the answers from other Christians were very unsatisfying. I used to believe that the Bible was, you know, inspired by a loving God, but its endorsement of genocide, the abuse of slaves, and the mistreatment of women and children really started to bother me.

I set out to study these issues as much as I could. I stayed up past midnight for weeks reading what Christians have to say, and this process triggered a real crisis of faith. What started out as a search for answers on Biblical genocide led me to places like commonsenseatheism.com. I learned that the Bible has serious credibility problems on lots of issues that no one ever told me about. Wow.

My Question

Now I'm pretty sure that the God of the Bible is man-made and Jesus of Nazareth was probably a failed prophet, but I don't have good reasons to reject the supernatural all together. I'm working through the sequences, but this process is slow. I will probably struggle with this question for months, maybe longer.

Excluding the Supernatural was interesting, but it left me wanting a more thorough explanation. Where do you think I should go from here? Should I just continue reading the sequences, and re-read them until the ideas gel? I'm coming from 30 years of Sunday School level thinking. It's not like I grew up with words like "epistemology" and "epiphenomenalism". If there is no supernatural, and I can be confident about that, I will need to re-evaluate a lot of things. My worldview is up for grabs.

• Yudkowsky, Reductionism sequence (2008)
• Ladyman et al., Every Thing Must Go: Metaphysics Naturalized (2007)
• Bickle, Philosophy and Neuroscience: A Ruthlessly Reductive Account (2003)
• Bickle, 'Real Reduction in Real Neuroscience' (2008)
• Glimcher, Foundations of Neuroeconomic Analysis (2010)
• Ney, 'Reductionism' [for Internet Encyclopedia of Philosophy] (2008)
• Hohwy & Kallestrup (eds.), Being Reduced (2008)
• Drescher, Good and Real (2006)

I can't endorse everything in all these works, but they each provide insights into understanding reduction.

What else do ya'll recommend?

Does building a computer count as explaining something to a rock?

(If we still had open threads, I would have posted this there. As it is, I figure this is better than not saying anything.)

Cross-posted from my LiveJournal:

Yesterday evening, I pasted to two IRC channels an excerpt of what someone had written. In the context of the original text, that excerpt had seemed to me like harmless if somewhat raunchy humor. What I didn't realize at the time was that by removing the context, the person writing it came off looking like a jerk, and by laughing at it I came off looking as something of a jerk as well.

Two people, both of whom I have known for many years now and whose opinions I value, approached me by private message and pointed out that that may not have been the smartest thing to do. My initial reaction was defensive, but I soon realized that they were right and thanked them for pointing it out to me. Putting on a positive growth mindset, I decided to treat this event as a positive one, as in the future I'd know better.

Later that evening, as I lay in bed waiting to fall asleep, the episode replayed itself in my mind. I learnt long ago that trying to push such replays out of my mind would just make them take longer and make them feel worse. So I settled back to just observing the replay and waiting for it to go away. As I waited, I started thinking about what kind of lower-level neural process this feeling might be a sign of.

Artificial neural networks use what is called a backpropagation algorithm to learn from mistakes. First the network is provided some input, then it computes some value, and then the obtained value is compared to the expected value. The difference between the obtained and expected value is the error, which is then propagated back from the end of the network to the input layer. As the error signal works it way through the network, neural weights are adjusted in such a fashion to produce a different output the next time.

Backprop is known to be biologically unrealistic, but there are more realistic algorithms that work in a roughly similar manner. The human brain seems to be using something called temporal difference learning. As Roko described it: "Your brain propagates the psychological pain 'back to the earliest reliable stimulus for the punishment'. If you fail or are punished sufficiently many times in some problem area, and acting in that area is always preceeded by [doing something], your brain will propagate the psychological pain right back to the moment you first begin to [do that something]".

As I lay there in bed, I couldn't help the feeling that something similar to those two algorithms was going on. The main thing that kept repeating itself was not the actual action of pasting the quote to the channel or laughing about it, but the admonishments from my friends. Being independently rebuked for something by two people I considered important: a powerful error signal that had to be taken into account. Their reactions filling my mind: an attempt to re-set the network to the state it was in soon after the event. The uncomfortable feeling of thinking about that: negative affect flooding the network as it was in that state, acting as a signal to re-adjust the neural weights that had caused that kind of an outcome.

After those feelings had passed, I thought about the episode again. Now I felt silly for committing that faux pas, for now it felt obvious that the quote would come across badly. For a moment I wondered if I had just been unusually tired, or distracted, or otherwise out of my normal mode of thought to not have seen that. But then it occurred to me - the judgment of this being obviously a bad idea was produced by the network that had just been rewired in response to social feedback. The pain of the feedback had been propagated back to the action that caused it, so just thinking about doing that (or thinking about having done that) made me feel stupid. I have no way of knowing whether the "don't do that, idiot" judgment is something that would actually have been produced had I been paying more attention, or if it's a genuinely new judgment that wouldn't have been produced by the old network.

I tend to be somewhat amused by the people who go about claiming that computers can never be truly intelligent, because a computer doesn't genuinely understand the information it's processing. I think they're vastly overestimating how smart we are, and that a lot of our thinking is just relatively crude pattern-matching, with various patterns (including behavioral ones) being labeled as good or bad after the fact, as we try out various things.

On the other hand, there would probably have been one way to avoid that incident. We do have the capacity for reflective thought, which allows us to simulate various events in our heads without needing to actually undergo them. Had I actually imagined the various ways in which people could interpret that quote, I would probably have relatively quickly reached the conclusion that yes, it might easily be taken as jerk-ish. Simply imagining that reaction might then have provided the decision-making network with a similar, albeit weaker, error signal and taught it not to do that.

However, there's the question of combinatorial explosions: any decision could potentially have countless of consequences, and we can't simulate them all. (See the epistemological frame problem.) So in the end, knowing the answer to the question of "which actions are such that we should pause to reflect upon their potential consequences" is something we need to learn by trial and error as well.

So I guess the lesson here is that you shouldn't blame yourself too much if you've done something that feels obviously wrong in retrospect. That decision was made by an earlier version of you. Although it feels obvious now, that version of you might literally have had no way of knowing that it was making a mistake, as it hadn't been properly trained yet.

A second attempt.

Defintions:

universe: that which contains everything.

reality: the realm of natural phenomena.

scientific theory: a theory that identifies natural phenomena.

morality: the realm of normative rules.

normative theory: a theory that identifies normative rules.

identification: "this natural phenomenon has following properties" or "this normative rule says: ... "

What are you?

Please answer in the form of [ABC0]{4}, where 0 stands for no opinion. Feel free to add an explanation.

Example: B0BA stands for anti-realism, no opinion on values, weak ontological realism, scientific reductionism.

1A realism

Reality is external to the mind.

It is possible to evaluate which scientific theory is more correct.

1B anti-realism

Reality is external to the mind.

It is impossible to evaluate which scientific theory is more correct.

1C subjectivism

Reality is a product of the mind.

2A value realism

Morality is external to the mind.

It is possible to evaluate which normative theory is better.

2B value anti-realism

Morality is external to the mind.

It is impossible to evaluate which normative theory is better.

2C value relativism

Morality is a product of the mind.

3A strong ontological reductionism

Mental phenomena are reducible to reality and reality is reducible to mathematics.

Mathematics is the universe.

3B weak ontological reductionism

Mental phenomena are reducible to reality, but reality is not reducible to mathematics.

Reality (and mathematics) is the universe.

3C anti-reductionism

Mental phenomena are not reducible to reality and reality is not reducible to mathematics.

4A scientific reductionism

The entirety of scientific theories can be reduced to some axiomatic theories.

4B scientific anti-reductionism

The entirety of scientific theories cannot be reduced to some axiomatic theories.

New natural phenomena require new irreducible scientific theories.

Since there is no handy toll to create polls on LW, please post comments on your position.

As which of the following would you identify yourself? (I am not good at rationalist taboo, thus please excuse me for ambiguous terms.)

Strong ontological reductionst

See defintion on Wikipedia. Someone who believes that mental phenomena can be fully reduced to physics and that physics can be fully reduced to mathematics. That is, desires and electrons don't have any fundamental qualities, but are in the end mathematical objects. And nothing exists outside the mathematical realm.

Weak ontological reductionist

Someone who believes that mental phenomena don't have any qualities outside the domain of physics. Every aspect of mental phenomena can be fully reduced to physical phenomena. But physical phenomena are not necessarily mathematical objects.

Strong scientific reductionist

Someone who believes that quantum mechanics is wrong and Laplace's demon can exist in principle (if unrestricted by physical limitations). 

Weak scientific reductionist

Someone who concedes that it is impossible in principle to predict complicated physical systems, but that the concepts and theories in chemistry and biology are mere approximations and simplifications of complicated physical computations to sidestep the (faster-than-)exponential wall. That is, chemical and biological models are not fundamental, but are reducible to physical theories (if we had the theoretical computational power to simulate the models).

Please also comment if you are not a reductionist and explain what kind of reductionist you are not.