On desiring subjective states (post 3 of 3)

7 torekp 05 May 2015 02:16AM

Carol puts her left hand in a bucket of hot water, and lets it acclimate for a few minutes.  Meanwhile her right hand is acclimating to a bucket of ice water.  Then she plunges both hands into a bucket of lukewarm water.  The lukewarm water feels very different to her two hands.  To the left hand, it feels very chilly.  To the right hand, it feels very hot.  When asked to tell the temperature of the lukewarm water without looking at the thermocouple readout, she doesn't know.  Asked to guess, she's off by a considerable margin.

water-hot-cold

Next Carol flips the thermocouple readout to face her (as shown), and practices.  Using different lukewarm water temperatures of 10-35 C, she gets a feel for how hot-adapted and cold-adapted hands respond to the various middling temperatures.  Now she makes a guess - starting with a random hand, then moving the other one and revising the guess if necessary - each time before looking at the thermocouple.  What will happen?  I haven't done the experiment, but human performance on similar perceptual learning tasks suggests that she will get quite good at it.

We bring Carol a bucket of 20 C water (without telling) and let her adapt her hands first as usual.  "What do you think the temperature is?" we ask.  She moves her cold hand first.  "Feels like about 20," she says.  Hot hand follows.  "Yup, feels like 20."

"Wait," we ask. "You said feels-like-20 for both hands.  Does this mean the bucket no longer feels different to your two different hands, like it did when you started?"

"No!" she replies.  "Are you crazy?  It still feels very different subjectively; I've just learned to see past that to identify the actual temperature."

In addition to reports on the external world, we perceive some internal states that typically (but not invariably) can serve as signals about our environment.  Let's tentatively call these states Subjectively Identified Aspects of Perception (SIAPs).  Even though these states aren't strictly necessary to know what's going on in the environment - Carol's example shows that the sensation felt by one hand isn't necessary to know that the water is 20 C, because the other hand knows this via a different sensation - they still matter to us.  As Eliezer notes:


If I claim to value art for its own sake, then would I value art that no one ever saw?  A screensaver running in a closed room, producing beautiful pictures that no one ever saw?  I'd have to say no.  I can't think of any completely lifeless object that I would value as an end, not just a means.  That would be like valuing ice cream as an end in itself, apart from anyone eating it.  Everything I value, that I can think of, involves people and their experiences somewhere along the line.

The best way I can put it, is that my moral intuition appears to require both the objective and subjective component to grant full value.


Subjectivity matters.  (I am not implying that Eliezer would agree with anything else I say about subjectivity.)

Why would evolution build beings that sense their internal states?  Why not just have the organism know the objective facts of survival and reproduction, and be done with it?  One thought is that it is just easier to build a brain that does both, rather than one that focuses relentlessly on objective facts.  But another is that this separation of sense-data into "subjective" and "objective" might help us learn to overcome certain sorts of perceptual illusion - as Carol does, above.  And yet another is that some internal states might be extremely good indicators and promoters of survival or reproduction - like pain, or feelings of erotic love.  This last hypothesis could explain why we value some subjective aspects so much, too.

Different SIAPs can lead to the same intelligent behavioral performance, such as identifying 20 degree C water.  But that doesn't mean Carol has to value the two routes to successful temperature-telling equally.  And, if someone proposed to give her radically different, previously unknown, subjectively identifiable aspects of experience, as new routes to the kinds of knowledge she gets from perception, she might reasonably balk.  Especially if this were to apply to all the senses.  And if the subjectively identifiable aspects of desire and emotion (SIADs, SIAEs) were also to be replaced, she might reasonably balk much harder.  She might reasonably doubt that the survivor of this process would be her, or even human, in any sense meaningful to her.

Would it be possible to have an intelligent being whose cognition of the world is mediated by no SIAPs?  I suspect not, if that being is well-designed.  See above on "why would evolution build beings that sense internal states."

If you've read all 3 posts, you've probably gotten the point of the Gasoline Gal story by now.  But let me go through some of the mappings from source to target in that analogy.  A car that, when you take it on a tour, accelerates well, handles nicely, makes the right amount of noise, and so on - one that passes the touring test (groan) - is like a being that can identify objective facts in its environment.  An internal combustion engine is like Carol's subjective cold-sensation in her left hand - one way among others to bring about the externally-observable behavior.  (By "externally observable" I mean "without looking under the hood".)  In Carol's case, that behavior is identifying 20 C water.  In the engine's case, it's the acceleration of the car.  Note that in neither case is this internal factor causally inert.  If you take it away and don't replace it with anything, or even if you replace it with something that doesn't fit, the useful external behavior will be severely impaired.  The mere fact that you can, with a lot of other re-working, replace an internal combustion engine with a fuel cell, does not even begin to show that the engine does nothing.

And Gasoline Gal's passion for internal combustion engines is like my - and I dare say most people's - attachment to the subjective internal aspects of perception and emotion that we know and love.  The words and concepts we use for these things - pain, passion, elation, for some easier examples - refer to the actual processes in human beings that drive the related behavior.  (Regarding which, neurology has more to learn.)  As I mentioned in my last post, a desire can form with a particular referent based on early experience, and remain focused on that event-type permanently.  If one constructs radically different processes that achieve similar external results, analogous to the fuel cell driven car, one gets radically different subjectivity - which we can only denote by pointing simultaneously to both the "under the hood" construction of these new beings, and the behavior associated with their SIAPs, together.

Needless to say, this complicates uploading.

One more thing: are SIAPs qualia?  A substantial minority of philosophers, or maybe a plurality, uses "qualia" in a sufficiently similar way that I could probably use that word here.  But another substantial minority loads it with additional baggage.  And that leads to pointless misunderstandings, pigeonholing, and straw men.  Hence, "SIAPs".  But feel free to use "qualia" in the comments if you're more comfortable with that term, bearing my caveats in mind.

The language of desire (post 2 of 3)

1 torekp 03 May 2015 09:57PM

To the extent that desires explain behavior, it is primarily by meshing with beliefs to favor particular actions.  For example, if I desire to lose 5 lbs, and I believe that exercising a half hour per day will cause me to lose 5 lbs, then this belief-desire pair makes it more likely that I will exercise.  Beliefs have semantic content.  In order to explain an action as part of a belief-desire pair, a desire must also have semantic content: one that in some sense "matches" a relevant part of the belief.  In the example, the matching semantic content is "to lose 5 lbs".

Of course, desires can also explain some behaviors without functioning as part of a belief-desire pair.  I might want something so badly, I start trembling.  No beliefs are required to explain the trembling.  Also, notably, desires usually (but not always) feel like something.  We gesture in the vague direction of these feelings by talking about "a burning desire", or "thirst" (for something that is not a drink), etc.  In these ways, "desire" is a richer concept than what I am really after, here.  That's OK, though; I'm not trying to define desire.  Alternatively, we can talk about "values", "goals", or "utility functions" - anything that interacts with beliefs, via semantics, to favor particular actions.  I will mostly stick to the word "desire", but nothing hangs on it.

So how does this "semantics" stuff work?

Let me start by pointing to the Sequences.  EY explains it pretty well, with some help by pragmatist in the comments.  Like EY, I subscribe to the broad class of causal theories of mental content.  For our purposes here, we need not choose among them.

The reference of the concepts involved in desires (or goals) is determined by their causal history.  To take a simple example, suppose Allie has a secret admirer.  It's Bob, but she doesn't know this.  Bob leaves her thoughtful gifts and love letters, which Allie appreciates so much that she falls in love with "my secret admirer".  She tells all her friends, "I can't wait to meet my secret admirer and have a torrid affair!"  Allie's desire refers to Bob, because Bob is the causal source of all the gifts and love letters, which in turn caused Allie's relevant thoughts and desires.

We could have told the story differently, in a way that made the reference of "my secret admirer" doubtful, or even hopeless.  We could have had many secret admirers, or maybe some pranksters, leaving different gifts and notes at different times, with Allie mistakenly attributing all to one source.  But that would be mean, and in this context pointless.  Let's not tell that story.

Kaj_Sotala brings up another point about desire, which I'd like to quote at length:


In most artificial RL [reinforcement learning] agents, reward and value are kept strictly separate. In humans (and mammals in general), this doesn't seem to work quite the same way. Rather, if there are things or behaviors which have once given us rewards, we tend to eventually start valuing them for their own sake. If you teach a child to be generous by praising them when they share their toys with others, you don't have to keep doing it all the way to your grave. Eventually they'll internalize the behavior, and start wanting to do it. One might say that the positive feedback actually modifies their reward function, so that they will start getting some amount of pleasure from generous behavior without needing to get external praise for it. In general, behaviors which are learned strongly enough don't need to be reinforced anymore (Pryor 2006).


A desire can form, with a particular referent based on early experience, and remain focused on that object or event-type permanently.  That's a point I will be making much hay of, in my third and final post in this mini-sequence.  I think it explains why Gasoline Gal's desire is not irrational - and neither are some desires many people have on my real target subject.

Gasoline Gal looks under the hood (post 1 of 3)

4 torekp 03 May 2015 08:15PM

"Hi," said Galaxy, "Are you Fuller Chen?  I'm here to see your 2018 Chevy SS.  Is it still for sale?"

"Sure," Fuller replied, "let me open the garage door for you." [Garage door opens.]  "What's your name?"

"I'm Galaxy, but everyone calls me Gal."  Galaxy walked around the car.  "Looks great.  Not a scratch or a blemish."

"Would you like to take it for a test drive?" Fuller asked, opening the passenger door for himself and reaching the key out to her.

"Yes, thanks," said Galaxy.  She adjusted her seat and mirrors, then started the car and eased it out of the garage and onto the residential street.  "Wow, this car is really quiet.  Is it a hybrid?  I didn't think they made a hybrid."

"Nope, not a hybrid," Fuller answered.  His smile made her think there was a joke in there somewhere.

She took it onto the main road and put the pedal to the metal.  The response was rapid and forceful - and eerily quiet.  "This car is too quiet," Galaxy objected.  "What's going on here?"  She slammed on the brakes and pulled into an empty parking lot.

"It's a fuel cell!" Fuller announced.  "Pop the hood, and take a look!  Retrofitted it myself.  It's a labor of love, but I need to cash in all my major assets for my business start-up.  That's why I'm letting it go at the price of an ordinary SS.  And as you have seen, the performance is equal to or better than any conventional engine."

Galaxy looked under the hood.  It sure did look like a fuel cell.  "We have a problem," Galaxy warned, "I admit, I am impressed with the performance, but ... I'm looking for a car with an internal combustion engine.  I need to hear the roar of the engine, feel the vibrations, and know that I'm being propelled by something that's literally exploding."

"Oh, if it's a roar you want, the car already has a speaker to generate noise for pedestrian safety," said Fuller.  "I could modify it to make the same audible sounds as a traditional internal combustion engine."

"No, listen:  my nickname is Gasoline Gal.  I grew up working in my father's auto repair shop.  I was the engine specialist.  Internal combustion is what I know and love.  It's what I want.  Not a simulacrum, but an actual internal combustion engine."

"But I can make this car behave the same as perceived from inside the cockpit as well as from outside the car.  You mentioned the vibrations of a regular engine.  It would be a simple matter to add a vibrating cam or two.  I tell you what, I'll make those modifications for free and you can test it by taking it on a tour.  If you can't tell the difference from the driver's seat, buy the car."

"No," Galaxy answered, "you're not getting it.  It's not about the behavior, it's about the underlying reality, the underlying cause.  I reject your test, and I insist on looking under the hood.  I love internal combustion engines.  Fuel cells are merely kinda cool."

"Is it about gasoline versus hydrogen?  Hydrogen is widely available nowadays.  I don't know why so many people are still prejudiced against it," said Fuller.

"No: it's not about the materials, it's about the internal processes.  And it's not that I'm against electrolytic energy conversion, it's that I'm wild about internal combustion."

"Hmm.  You came to love those engines by working with them and getting to know them.  Maybe if you worked with fuel cells, you could come to feel the same way about them," Fuller suggested.

"Maybe," Galaxy conceded, "but I'm not willing to wait and see.  I'm not looking for motivational reform.  I want what I want."

"But maybe your desire is irrational," Fuller suggested.  "Are you sure that driving an internal combustion powered car is really a terminal value for you?"

"Heck if I know!" Gasoline Gal answered. "But it is already clear that this car, despite its acceleration, reliability, style, and reasonable price, just doesn't do it for me.  I need to go find a good old-fashioned car to buy.  One with an actual internal combustion engine."

And the next day, she did.  She paid a little more and the acceleration was a tiny bit less, but the acceleration came from the right source.

----

The Question: Is there good reason to suppose that Gal's desire for internal combustion is irrational, and if so, where's her mistake?

The story is an analogue, along a narrow dimension, to my real subject.  I flirted with a horrific pun in there - bonus points for pointing it out - which reveals it.  But you might want to save the inevitable critique of the analogy for later.  I plan on two more posts.  In my next post I'll try to say something about the language (semantics) of desire; then in a third post I'll lay out the analogy.

[LINK] Prisoner's Dilemma? Not So Much

4 torekp 20 May 2014 11:38PM

Hannes Rusch argues that the Prisoner's Dilemma is best understood as merely one game of very many:

only 2 of the 726 combinatorially possible strategically unique ordinal 2x2 games have the detrimental characteristics of a PD and that the frequency of PD-type games in a space of games with random payoffs does not exceed about 3.5%. Although this does not compellingly imply that the relevance of PDs is overestimated, in the absence of convergent empirical information about the ancestral human social niche, this finding can be interpreted in favour of a rather neglected answer to the question of how the founding groups of human cooperation themselves came to cooperate: Behavioural and/or psychological mechanisms which evolved for other, possibly more frequent, social interaction situations might have been applied to PD-type dilemmas only later.

http://www2.units.it/etica/2013_2/RUSCH.pdf

Robots ate my job [links]

5 torekp 10 April 2012 01:57AM

Mechanical Engineering magazine (paywalled until next month) and Financial Times, among others, recently reviewed the book Race Against the Machine by economists Erik Brynjolfsson and Andrew McAfee.  The FT reviewer writes:

Pattern recognition, the authors think, will quickly allow machines to branch out further. Computers will soon drive more safely than humans, a fact Google has demonstrated by allowing one to take out a Toyota Prius for a 1,000-mile spin. Truck and taxi drivers should be worried – but then so should medical professionals, lawyers and accountants; all of their jobs are at risk too. The outcome is a nightmarish but worryingly convincing vision of a future in which an ever-decreasing circle of professions is immune from robotic encirclement.

And ME magazine quotes McAfee in an interview:

Once computers get better than people, you don't have to hire people to do that job any more.  That doesn't mean that people can't find work.  There will always be an amount of work to do, but they won't like the wages they are offered.

Both reviewers also hint that McAfee and Brynjolfsson offer a partial explanation of the "jobless recovery", but either the book's argument is weak or the reviewers do a poor job summarizing it.  Such a purported explanation might be the main attraction for most readers, but I'm more interested in the longer-term picture.  Be it the "nightmarish vision" of the future mentioned in FT, or the simpler point about wages offered by McAfee, this might be a good hook to get the general public thinking about the long-term consequences of AI.

Is that a good idea?  Should sleeping general publics be left to lie?  There seems to be significant reluctance among many LessWrongers to stir the public, but have we ever hashed out the reasons for and against?  Please describe any non-obvious reasons on either side.