You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

TheAncientGeek comments on The AI in Mary's room - Less Wrong Discussion

4 Post author: Stuart_Armstrong 24 May 2016 01:19PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (58)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheAncientGeek 27 May 2016 06:15:16PM 1 point [-]

M's R is about what it says its about, the existence of non physical facts. Finding a loophole where Mary can instantiate the brain state without having the perceptual stimulus doesn't address that...indeed it assumes that an instantiation of the red-seeing is necessary, which is tantamount to conceding that something subectve is going on, which is tantamount to conceding the point.

Comment author: ShardPhoenix 29 May 2016 01:29:27AM *  1 point [-]

non physical facts

What is a "non physical fact"? The experience of red seems to be physically encoded in the brain like anything else. It does seem clear that some knowledge exists which can't be transmitted from human to human via means of language, at least not in the same way that 2+2=4 can. However, this is just a limitation of the human design that doesn't necessarily apply to eg AIs (which depending on design may be able to transmit and integrate snippets of their internal code and data), and I don't think this thought experiment proves anything beyond that.

Comment author: TheAncientGeek 29 May 2016 09:13:17AM *  -1 points [-]

What is a "non physical fact"?

The argument treats physical knowledge as a subset of objective. kowledge. Subjective knowledge, which can only be known on a first person basis, automatically counts as non physical. That's an epistemic definition.

The experience of red seems to be physically encoded in the brain like anything else.

If you have the expected intuition from M's R, that Mary would be able to read cognitive information from brain scans, but not expetuental information. In that send, 'red' is not encoded in the same way as everything else, since it can not be decoded in the same way.

sIt does seem clear that some knowledge exists which can't be transmitted from human to human via means of language, at least not in the same way that 2+2=4 can. However, this is just a limitation of the human design

But noit super human design. The original paper (ave you read it?) avoids the issue of limited communication bandwidth by making Mary a super scientist who can examine brain scans of any level of detail.

Proves anything beyond that

What it proves to you depends on what intuitions you have about it . If you think Mary would know what red looks like while in the room, from reading brain scans, then it s going to prove anything to you.

Comment author: ShardPhoenix 29 May 2016 11:15:02AM *  0 points [-]

A way to rephrase the question is, "is there any sequence of sensory inputs other than the stimulation of red cones by red light that will cause Mary to have comparable memories re: the color red as someone who has had their red cones stimulated at some point". It's possible that the answer is no, which says something interesting about the API of the human machine, but doesn't seem necessarily fundamental to the concept of knowledge.

Comment author: TheAncientGeek 29 May 2016 12:50:43PM *  -2 points [-]

The relevance is physicalism.

If physicalism is the claim that everything, has a physical explanation, then the inability to understand what pain is without being in pain is a contradiction to it. I don' think anyone here believes that physicalism is an unmportamt issue.

Comment author: ShardPhoenix 29 May 2016 01:36:54PM 0 points [-]

I'm arguing that there's no contradiction and that this inability is just a limit of humans/organic brains, not a fundamental fact about pain or information.

Comment author: TheAncientGeek 29 May 2016 03:12:51PM *  0 points [-]

If you want to argue to that conclusion, then arge for it: What kind of limit? Where does it come from?

Comment author: ImNotAsSmartAsIThinK 28 May 2016 12:33:30AM 0 points [-]

I think the argument is asserting that Mary post-brain surgery is a identical to Mary post-seeing-red. There is no difference; the two Mary's would both attest to having access some ineffable quality of red-ness.

To put it bluntly, both Marys say the same things, think the same things, and generally are virtually indistinguishable. I don't understand what disagreement is occurring here, hopefully I've given someone enough ammunition to explain.

Comment author: TheAncientGeek 28 May 2016 07:21:13AM 0 points [-]

I don't understand what the point of that point is.

Do you think you argung against the intended conclusion of the Knowledge Argumemt in some way? If so, you are not...the loophole you have found s quite irrelevant,

Comment author: ImNotAsSmartAsIThinK 28 May 2016 03:10:03PM 1 point [-]

I have no idea what your position even is and you are making no effort to elucidate it. I had hoped this line

I don't understand what disagreement is occurring here, hopefully I've given someone enough ammunition to explain.

Was enough to clue you in to the point of my post.

Comment author: TheAncientGeek 28 May 2016 04:44:16PM *  0 points [-]

I'm disagreeing that you have a valid refutation of the KA. However, I don't know if you even think you have, since you haven't responded to my hints that you should clarify.

Comment author: ImNotAsSmartAsIThinK 28 May 2016 10:07:16PM 0 points [-]

"I think you're wrong" is not a position.

They way you're saying this, it makes it seem like we're both in the same boat. I have no idea what position you're even holding.

I feel like I'm doing the same thing over and over and nothing different is happening, but I'll quote what I said in another place in this thread and hope I was a tiny bit clearer.

http://lesswrong.com/lw/nnc/the_ai_in_marys_room/day2

I think the distinction between 'knowing all about' and 'seeing' red is captured in my box analogy. The brain state is a box. There is another box inside it, call this 'understanding'. We call something inside the first box 'experienced'. So the paradox hear is the two distinct states [experiencing (red) ] and [experiencing ( [understanding (red) ] ) ] are both brought under the header [knowing (red)], and this is really confusing.

Comment author: TheAncientGeek 29 May 2016 07:51:54AM *  0 points [-]

I just explained the position I am holding.

"I think you're wrong" is not a position

I have explained elsewhere why the loophole doesnt work:-

M's R is about what it says its about, the existence of non physical facts. Finding a loophole where Mary can instantiate the brain state without having the perceptual stimulus doesn't address that...indeed it assumes that an instantiation of the red-seeing is necessary, which is tantamount to conceding that something subectve is going on, which is tantamount to conceding the point

Moving on to your argument:-

I think the distinction between 'knowing all about' and 'seeing' red is captured in my box analogy. The brain state is a box. There is another box inside it, call this 'understanding'. We call something inside the first box 'experienced'. So the paradox hear is the two distinct states [experiencing (red) ] and [experiencing ( [understanding (red) ] ) ] are both brought under the header [knowing (red)], and this is really confusing

Confusing to whom?

Let's suppose that person is Frank Jackson.

In the knowledge Argument, Jackson credits Mary with all objective knowledge, and only objective knowledge precisely because he is trying to establish the existence of subjective knowledge .. what Mary doesnt know must be subjective, if there is something Mary doesn't know. So the eventual point s that there s more to knowledge than objective knowledge.

So you don't show that Jackson is wrong by agreeing with him.

But I don't know that you think Jackson is wrong,

Comment author: ImNotAsSmartAsIThinK 29 May 2016 07:28:03PM *  0 points [-]

Mary's room seems to be arguing that,

[experiencing(red)] =/= [experiencing(understanding([experiencing(red)] )] )]

(translation: the experience of seeing red is no the experience of understanding how seeing red works)

This is true, when we take those statements literally. But it's true in the same sense a Gödel encoding of statement in PA is not literally that statement. It is just a representation, but the representation is exactly homomorphic to its referent. Mary's representation of reality is presumed complete ex hypothesi, therefore she will understand exactly what will happen in her brain after seeing color, and that is exactly what happens.

You wouldn't call a statement of PA that isn't a literally a Gödel encoding of a statement (for some fixed encoding) a non-mathematical statement. For one, because that statement has a Gödel encoding by necessity. But more importantly, even though the statement technically isn't literally a Gödel-encoding, it's still mathematical, regardless.

Mary's know how she will respond to learning what red is like. Mary knows how others will respond. This exhausts the space of possible predictions that could be made on behalf of this subjective knowledge, and it can be done without it.

what Mary doesnt know must be subjective, if there is something Mary doesn't know. So the eventual point s that there s more to knowledge than objective knowledge.

Tangentially to this discussion, but I don't think that is a wise way of labeling that knowledge.

Suppose Mary has enough information to predict her own behavior. Suppose she predicts she will do x. Could she not, upon deducing that fact, decide to not do x?

Mary has all objective knowledge, but certain facts about her own future behavior must escape her, because any certainty could trivially be negated.

Comment author: TheOtherDave 30 May 2016 03:48:20AM 0 points [-]

Suppose Mary has enough information to predict her own behavior. Suppose she predicts she will do x. Could she not, upon deducing that fact, decide to not do x?

There are three possibilities worth disambiguating here.
1) Mary predicts that she will do X given some assumed set S1 of knowledge, memories, experiences, etc., AND S1 includes Mary's knowledge of this prediction.
2) Mary predicts that she will do X given some assumed set S2 of knowledge, memories, experiences, etc., AND S2 does not include Mary's knowledge of this prediction.
3) Mary predicts that she will do X independent of her knowledge, memories, experiences, etc.

Comment author: TheAncientGeek 01 June 2016 09:34:23AM *  0 points [-]

Mary's representation of reality is presumed complete ex hypothesi, therefore she will understand exactly what will happen in her brain after seeing color, and that is exactly what happens

Mary is presumed to have all objective knowedge and only objectve knowledge, Your phrasing is ambiguous and therefire doesnt address the point. When you say Mary will know what happens when she sees red, do you mean she knows how red looks subjectively, or she knows something objective like what her behaviour will be.....further on you mention predicting her reactions,

You wouldn't call a statement of PA that isn't a literally a Gödel encoding of a statement (for some fixed encoding) a non-mathematical statement. For one, because that statement has a Gödel encoding by necessity. But more importantly, even though the statement technically isn't literally a Gödel-encoding, it's still mathematical, regardless.

Is that supposed to relate to the objective/ subjective distinction somehow?

[Mary knows] how she will respond to learning what red is like. Mary knows how others will respond. This exhausts the space of possible predictions that could be made on behalf of this subjective knowledge, and it can be done without

So? The overall point is about physicalism, and to get to 'physicalism is false', all you need is the existence of subjective knowledge, not its usefulness in making prediction. So again I don't see the relevance

Suppose Mary has enough information to predict her own behavior. Suppose she predicts she will do x. Could she not, upon deducing that fact, decide to not do x?

Maybe. I don't see the problem. There is still an unproblematic sense in which Mary has all objective knowledge, even if it doesn't allow her to do certain things. If that was the point.

Comment author: ImNotAsSmartAsIThinK 02 June 2016 09:16:44PM *  0 points [-]

Mary is presumed to have all objective knowedge and only objectve knowledge, Your phrasing is ambiguous and therefire doesnt address the point.

The behavior of the neurons in her skull is an objective fact, and this is what I mean to referring to. Apologies for the ambiguity.

When you say Mary will know what happens when she sees red, do you mean she knows how red looks subjectively, or she knows something objective like what her behaviour will be

The latter. The former is purely experiential knowledge, and as I have repeatedly said is contained in a superset of verbal (what you call 'objective') knowledge, but is disjoint with the set of verbal ('objective') knowledge itself. This is my box metaphor.

Is that supposed to relate to the objective/ subjective distinction somehow?

Yes. Assuming the Godel encoding is fixed, [the metaphor is that] any and all statements of PA are experiential knowledge (an experience, in simple terms), non-Godel statements of PA are purely experiential knowledge; the redness of red, say, and finally the Godel statements of PA are verbal knowledge, or 'objective knowledge' in your terminology.

Despite not being Godel statements in the encoding, the second item in the above list is still mathematical, and redness of red is still physical.

So? The overall point is about physicalism, and to get to 'physicalism is false', all you need is the existence of subjective knowledge, not its usefulness in making prediction. So again I don't see the relevance

What does this knowledge do? How do we tell the difference between someone with and without these 'subjective experiences'? What definition of knowledge admits it as valid?

Comment author: entirelyuseless 01 June 2016 01:48:04PM -1 points [-]

If one of Mary's predictions is, "When I see red, I will say, 'wow! I didn't know that red looked like that!", then the fact that she has predicted this in advance is hardly evidence that she does not learn anything by seeing red. If anything, it proves that she does.