You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

The AI in Mary's room

4 Stuart_Armstrong 24 May 2016 01:19PM

In the Mary's room thought experiment, Mary is a brilliant scientist in a black-and-white room who has never seen any colour. She can investigate the outside world through a black-and-white television, and has piles of textbooks on physics, optics, the eye, and the brain (and everything else of relevance to her condition). Through this she knows everything intellectually there is to know about colours and how humans react to them, but she hasn't seen any colours at all.

After that, when she steps out of the room and sees red (or blue), does she learn anything? It seems that she does. Even if she doesn't technically learn something, she experiences things she hadn't ever before, and her brain certainly changes in new ways.

The argument was intended as a defence of qualia against certain forms of materialism. It's interesting, and I don't intent to solve it fully here. But just like I extended Searle's Chinese room argument from the perspective of an AI, it seems this argument can also be considered from an AI's perspective.

Consider a RL agent with a reward channel, but which currently receives nothing from that channel. The agent can know everything there is to know about itself and the world. It can know about all sorts of other RL agents, and their reward channels. It can observe them getting their own rewards. Maybe it could even interrupt or increase their rewards. But, all this knowledge will not get it any reward. As long as its own channel doesn't send it the signal, knowledge of other agents rewards - even of identical agents getting rewards - does not give this agent any reward. Ceci n'est pas une récompense.

This seems to mirror Mary's situation quite well - knowing everything about the world is no substitute from actually getting the reward/seeing red. Now, a RL's agent reward seems closer to pleasure than qualia - this would correspond to a Mary brought up in a puritanical, pleasure-hating environment.

Closer to the original experiment, we could imagine the AI is programmed to enter into certain specific subroutines, when presented with certain stimuli. The only way for the AI to start these subroutines, is if the stimuli is presented to them. Then, upon seeing red, the AI enters a completely new mental state, with new subroutines. The AI could know everything about its programming, and about the stimulus, and, intellectually, what would change about itself if it saw red. But until it did, it would not enter that mental state.

If we use ⬜ to (informally) denote "knowing all about", then ⬜(X→Y) does not imply Y. Here X and Y could be "seeing red" and "the mental experience of seeing red". I could have simplified that by saying that ⬜Y does not imply Y. Knowing about a mental state, even perfectly, does not put you in that mental state.

This closely resembles the original Mary's room experiment. And it seems that if anyone insists that certain features are necessary to the intuition behind Mary's room, then these features could be added to this model as well.

Mary's room is fascinating, but it doesn't seem to be talking about humans exclusively, or even about conscious entities.

Comments (58)

Maybe we can perform the "Mary's Room" thought experiment

5 DavidPlumpton 14 April 2015 09:19AM

It seems possible that soon there may be a cure for colourblindness. The Mary's Room thought experiment attempts to pin down something about the nature of qualia in a contrived but similar situation, but my feeling is that the actual result of such an experiment would not be obvious. Would we consider the experiment valid if it was performed on somebody familiar with blue and green, but not red?

Comments (7)

Probability of coming into existence again ?

5 pzwczzx 28 February 2015 12:02PM

This question has been bothering me for a while now, but I have the nagging feeling that I'm missing something big and that the reasoning is flawed in a very significant way. I'm not well read in philosophy at all, and I'd be really surprised if this particular problem hasn't been addressed many times by more enlightened minds. Please don't hesitate to give reading suggestions if you know more. I don't even know where to start learning about such questions. I have tried the search bar but have failed to find a discussion around this specific topic.

I'll try and explain my train of thought as best as I can but I am not familiar with formal reasoning, so bear with me! (English is not my first language, either)

Based on the information and sensations currently available, I am stuck in a specific point of view and experience specific qualia. So far, it's the only thing that has been available to me; it is the entirety of my reality. I don't know if the cogito ergo sum is well received on Less Wrong, but it seems on the face of it to be a compelling argument for my own existence at least.

Let's assume that there are other conscious beings who "exist" in a similar way, and thus other possible qualia. If we don't assume this, doesn't it mean that we are in a dead end and no further argument is possible? Similar to what happens if there is no free will and thus nothing matters since no change is possible? Again, I am not certain about this reasoning but I can't see the flaw so far.

There doesn't seem to be any reason why I should be experiencing these specific qualia instead of others, that I "popped into existence" as this specific consciousness instead of another, or that I perceive time subjectively. According to what I know, the qualia will probably stop completely at some subjective point in time and I will cease to exist. The qualia are likely to be tied to a physical state of matter (for example colorblindness due to different cells in the eyes) and once the matter does not "function" or is altered, the qualia are gone. It would seem that there could be a link between the subjective and some sort of objective reality, if there is indeed such a thing.

On a side note, I think it's safe to ignore theism and all mentions of a pleasurable afterlife of some sort. I suppose most people on this site have debated this to death elsewhere and there's no real point in bringing it up again. I personally think it's not an adequate solution to this problem.

Based on what I know, and that qualia occur, what is the probability (if any) that I will pop into existence again and again, and experience different qualia each time, with no subjectively perceivable connection with the "previous" consciousness? If it has happened once, if a subjective observer has emerged out of nothing at some point, and is currently observing subjectively (as I think is happening to me), does the subjective observing ever end?

I know it sounds an awful lot like mysticism and reincarnation, but since I am currently existing and observing in a subjective way (or at least I think I am), how can I be certain that it will ever stop?

The only reason why this question matters at all is because suffering is not only possible but quite frequent according to my subjective experience and my intuition of what other possible observers might be experiencing if they do exist in the same way I do. If there were no painful qualia, or no qualia at all, nothing would really matter since there would be no change needed and no concept of suffering. I don't know how to define suffering, but I think it is a valid concept and is contained in qualia, based on my limited subjectivity.

This leads to a second, more disturbing question : does suffering have a limit or is it infinite? Is there a non zero probability to enter into existence as a being that experiences potentially infinite suffering, similar to the main character in I have no mouth and I must scream? Is there no way out of existence? If the answer is no, then how would it be possible to lead a rational life, seeing as it would be a single drop in an infinite ocean?

On a more positive note, this reasoning can serve as a strong deterrent to suicide, since it would be rationally better to prolong your current and familiar existence than to potentially enter a less fortunate one with no way to predict what might happen.

Sadly, these thoughts have shown to be a significant threat to motivation and morale. I feel stuck in this logic and can't see a way out at the moment. If you can identify a flaw here, or know of a solution, then I eagerly await your reply.

kind regards

 

 

 

Comments (11)

The raw-experience dogma: Dissolving the “qualia” problem

2 metaphysicist 16 September 2012 07:15PM

[Cross-posted.]

1. Defining the problem: The inverted spectrum

Philosophy has been called a preoccupation with the questions entertained by adolescents, and one adolescent favorite concerns our knowledge of other persons’ “private experience” (raw experience or qualia). A philosophers’ version is the “inverted spectrum”: how do I know you see “red” rather than “blue” when you see this red print? How could we tell when we each link the same terms to the same outward descriptions? We each will say “red” when we see the print, even if you really see “blue.”

The intuition that allows us to be different this way is the intuition of raw experience (or of qualia). Philosophers of mind have devoted considerable attention to reconciling the intuition that raw experience exists with the intuition that inverted-spectrum indeterminacy has unacceptable dualist implications making the mental realm publicly unobservable, but it’s time for nihilism about qualia, whose claim to exist rests solely on the strength of a prejudice.

A. Attempted solutions to the inverted spectrum.

One account would have us examine which parts of the brain are activated by each perception, but then we rely on an unverifiable correlation between brain structures and “private experience.” With only a single example of private experience—our own—we have no basis for knowing what makes private experience the same or different between persons.

A subtler response to the inverted spectrum is that red and blue as experiences are distinct because red looks “red” due to its being constituted by certain responses, such as affect. Red makes you alert and tense; blue, tranquil or maybe sad. What we call the experience of red, on this account, just is the sense of alertness, and other manifestations. The hope is that identical observable responses to appropriate wavelengths might explain qualitative redness. Then, we could discover we experience blue when others experience red by finding that we idiosyncratically become tranquil instead of alert when exposed to the long wavelengths constituting physical red. This complication doesn’t remove the radical uncertainty about experiential descriptions. Emotion only seems more capable than cognition of explaining raw experience because emotional events are memorable. The affect theory doesn't answer how an emotional reaction can constitute a raw subjective experience.

B. The “substitution bias” of solving the “easy problem of consciousness” instead of the “hard problem.”

As in those examples, attempts at analyzing raw experience commonly appeal to the substitution process that psychologist Daniel Kahneman discovered in many cognitive fallacies. Substitution is the unthoughtful replacement of an easy for a related hard question. In the philosophy of mind, the distinct questions are actually termed the “easy problem of consciousness” and the “hard problem of consciousness,” and errors regarding consciousness typically are due to substituting the “easy problem” for the “hard,” where the easy problem is to explain some function that typically accompanies “awareness.” The philosopher might substitute knowledge of one’s own brain processes for raw experience; or, as in the previous example, experience’s neural accompaniments or its affective accompaniments. Avoiding the “substitution bias” is particularly hard when dealing with raw awareness, an unarticulated intuition; articulating it is a present purpose.

2. The false intuition of direct awareness

A. Our sense that the existence of raw experience is self-evident doesn’t show that it is true.

The theory that direct awareness reveals raw experience has long been almost sacrosanct in philosophy. According to the British Empiricists, direct experience consists of sense data and forms the indubitable basis of all synthetic knowledge. For Continental Rationalist Descartes, too, my direct experience—“I think”—indubitably proves my existence.
We do have a strong intuition that we have raw experience, the substance of direct awareness, but we have other strong intuitions, some turn out true and others false. We have an intuition that space is necessarily flat, an intuition proven false only with non-Euclidean geometries in the 19th century. We have an intuition that every event has a cause, which determinists believe but indeterminists deny. Sequestered intuitions aren’t knowledge.

B. Experience can’t reveal the error in the intuition that raw experience exists.

To correct wayward intuitions, we ordinarily test them against each other. A simple perceptual illusion illustrates: the popular Muller-Lyer illusion, where arrowheads on a line make it appear shorter than an identical line with the arrowheads reversed. Invoking the more credible intuition that measuring the lines finds their real length convinces us of the intuitive error that the lines are unequal. In contrast, we have no means to check the truth of the belief in raw experience; it simply seems self-evident, but it might seem equally self-evident if it were false. 

C. We can’t capture the ineffable core of raw experience with language because there’s really nothing there.

One task in philosophy is articulating the intuitions implicit in our thinking, and sometimes rejecting the intuition should result from concluding it employs concepts illogically. What shows the intuition of raw experience is incoherent (self-contradictory or vacuous) is that the terms we use to describe raw experience are limited to the terms for its referents; we have no terms to describe the experience as such, but rather, we describe qualia by applying terms denoting the ordinary cause of the supposed raw experience. The simplest explanation for the absence of a vocabulary to describe the qualitative properties of raw experience is that they don’t exist: a process without properties is conceptually vacuous.

D. We believe raw experience exists without detecting it.

One error in thinking about the existence of raw experience comes from confusing perception with belief, which is conceptually distinct. When people universally report that qualia “seem” to exist, they are only reporting their beliefs—despite their sense of certainty. Where “perception” is defined as a nervous system’s extraction of a sensory-array’s features, people can’t report their perceptions except through beliefs the perceptions sometimes engender: I can’t tell you my perceptions except by relating my beliefs about them. This conceptual truth is illustrated by the phenomenon of blindsight, a condition in  patients report complete blindness yet, by discriminating external objects, demonstrate that they can perceive them. Blindsighted patients can report only according to their beliefs, and they perceive more than they believe and report that they perceive. Qualia nihilism analyzes the intuition of raw experience as perceiving less than you believe and report you perceive, the reverse of blindsight.

3. The conceptual economy of qualia nihilism pays off in philosophical progress

Eliminating raw experience from ontology produces conceptual economy. A summary of its conceptual advantages:

   A. Qualia nihilism resolves an intractable problem for materialism: physical concepts are dispositional, whereas raw experiences concern properties that seem, instead, to pertain to noncausal essences. If raw experience was coherent, we could hope for a scientific insight, although no one has been able to define the general character of such an explanation. Removing a fundamental scientific mystery is a conceptual gain.
 
    B. Qualia nihilism resolves the private-language problem. There seems to be no possible language that uses nonpublic concepts. Eliminating raw experience allows explaining the absence of a private language by the nonexistence of any private referents.

    C.  Qualia nihilism offers a compelling diagnosis of where important skeptical arguments regarding the possibility of knowledge go wrong. The arguments—George Berkeley’s are their prototype—reason that sense data, being indubitable intuitions of direct experience, are the source of our knowledge, which must, in consequence, be about raw experience rather than the “external world.” If you accept the existence of raw experience, the argument is notoriously difficult to undermine logically because concepts of “raw experience” truly can’t be analogized to any concepts applying to the external world. Eliminating raw experience provides an effective demolition; rather than the other way around, our belief in raw experience depends on our knowledge of the external world, which is the source of the concepts we apply to fabricate qualia.

4. Relying on the brute force of an intuition is rationally specious.

Against these considerations, the only argument for retaining raw experience in our ontology is the sheer strength of everyone’s belief in its existence. How much weight should we attach to a strong belief whose validity we can't check? None. Beliefs ordinarily earn a presumption of truth from the absence of empirical challenge, but when empirical challenge is impossible in principle, the belief deserves no confidence.

Comments (340)

Rationalist sonnets about being a person

6 NancyLebovitz 23 March 2011 01:43AM

7 sonnets and some comments

A friend tells me that, if there is no soul,
There is no clash of body against mind.
I hate to be contentious, but I find
The case is rather different, on the whole.

For flesh and mind are clashing all the time;
The flesh says "eat!", the mind says "lose some weight."
The mind cries "run!", the flesh drones "vegetate,"
The soul is no wise guilty of this crime.

Am I the athlete who desires to run,
Or else the slugabed who yearns for quiet?
Do I crave food, or would I rather diet?
The I that speaks is both, and neither one.

When flesh and mind contend with shouts obscene
I place the soul--the self--smack in between.

                                         -----smallship1

 

The body and the mind are the two hands
that weave the self between them, interplay
a dialogue that may change day to day
creates consistency. Self understands

what neither flesh nor mind can apprehend
yet is a fiction and a referee
yet needs to be reined in. So fluently
its guesses become fantasies and end

in things we cannot know, that are not there
-God, Hell and Heaven - all ways to deny
the simple tasks life gives us. Mortify
the flesh, confuse the mind. Hope and despair.

The self's a servant. Use it, never let
it rule, or you will die full of regret.

                                  -----rozk

 

 

:

Comments (6)

A writer describes gradually losing language

12 NancyLebovitz 08 November 2010 03:57PM

A writer's memoir of a brain tumor slowly destroying his ability to use language

 

When I came to read this passage "…floating and flailing weightlessly.…" I said the word "weightlessly" as "walterkly". It took quite a bit of effort to be fully sure that this was a mistake; and more effort and repeating to grasp what exactly this nonsense word was, to establish its sound – I had to construct it phoneme by phoneme – clearly enough to write it down. And it seems that the reading eye, darting backwards and forwards, was plucking letters from the whole vicinity, and mixing them up, having lost its usual ability to sort them.What the whole thing emphasises, of course, is how what we call self-command is really a matter of having reliable automatic mechanisms, unthinking habits or instincts.
Comments (1)
Powered by Reddit Powered by Reddit