Yvain comments on Would Your Real Preferences Please Stand Up? - Less Wrong

42 Post author: Yvain 08 August 2009 10:57PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (131)

You are viewing a single comment's thread. Show more comments above.

Comment author: RobinHanson 10 August 2009 03:22:19PM 2 points [-]

The words "conscious" and "unconscious" are widely used; I don't think it helps for you to make up your own definitions. Your evidence about the rationality and morality of your conscious mind come many from your personal conscious beliefs about those features within yourself; these could easily be biased and self-serving. I'm still not entirely clear on what qualia mean, but from what I do understand about them I don't see why the parts of your mind other than the part I'm talking to couldn't have them.

Comment author: Yvain 10 August 2009 07:14:47PM *  5 points [-]

I thought I was listing the standard definition of "conscious" and "unconscious" in a way that made it clearer why they led to my conclusion. If you have a different definition, what is it?

My beliefs about the rationality and morality of my unconscious mind do come from my conscious mind, this is where all my beliefs come from. When I say that a creationist is less rational than I am, or a Nazi is less moral than I am, I'm using those same beliefs from my conscious mind, and they are subject to the same biases. I have to either forfeit my right to judge entirely, or use those same judgments to judge my unconscious. I've learned (partly from you) ways to try to be less biased, and I try to use them here, but in the end all I have is reflective coherence

Just to clarify your position, are you suggesting I have a moral duty to respect my unconscious mind's preferences, in the same way I would have to respect the preferences of another person? Or are you suggesting it would benefit my conscious mind to have inner peace?

Comment author: RobinHanson 10 August 2009 07:52:35PM 1 point [-]

So to summarize, you think your conscious mind is more rational, moral, and qualia-full than your unconscious mind, and the evidence you cite for these conclusions is: your conscious mind has these opinions. Have I got this right? Any idea what opinions your unconscious mind has on these matters?

Comment author: Yvain 10 August 2009 08:17:59PM *  7 points [-]

You can phrase any argument that way:

"You think you're more likely to be correct about evolution than a creationist, and the evidence you cite for this conclusion is that this is your opinion."

Yes, it's my opinion. But it's my opinion for reasons. I thought I gave some good reasons why I thought my conscious mind makes better decisions than my unconscious. You rejected those because it was my conscious mind giving them. But if that were sufficient criteria to reject reasons, you would have to reject everyone's reasons on any subject, from evolution to the roundness of the Earth.

Even aside from all those reasons, I haven't heard any reasons to think the unconscious is aware, rational, or morally reflective, and I think the burden of proof is on that position, since it would basically be saying there's a second person inside my head.

As for what opinion my unconscious mind has, yes, I have some idea. I predict it has no opinions at all, and is more of a collection of drives, instincts, and processes than the sort of entity that forms rational opinions on complicated issues. I doubt my unconscious "disagrees" with me about this any more than my kidneys disagree with my position on tax reform, or my tibia disagrees with my interpretation of quantum mechanics.

If I had multiple personality/dissociative identity disorder, I would be prepared to treat my alternate personalities as worthy of respect and cooperation. But I think my unconscious is probably more like a kidney or a tibia than like a whole other personality. I realize this is a factual claim, and am willing to change my mind if I hear evidence that suggests otherwise.

Comment author: RobinHanson 12 August 2009 02:35:55AM 1 point [-]

I didn't see you offering reasons - I just saw you declaring that in general the conscious is more rational and moral, this conclusion being so obvious it didn't need reasons. You later gave specific examples of beliefs in yourself where your conscious part thinks your conscious beliefs are more correct and more moral than your unconscious beliefs, but surely you can't expect that to be considered a sufficient argument about the general trend in all people on all topics.

I do think you could stand to read a bit more about the unconscious; I think you will find it far more complex and capable than you realize.

Comment author: Yvain 12 August 2009 04:23:30AM 8 points [-]

Things I gave as evidence: the logical inconsistency of unconscious mind having conscious experience, irrationality of unconscious mind continuing to pursue subgoals when clearly no longer connected to supergoals, unconscious' vulnerability to proximity/scale biases when dealing with morality, and several others. I don't see how any of these can be dismissed as just "my conscious part thinks my conscious beliefs are more correct" with anything other than a Fully General Counterargument.

I've read plenty about the unconscious, and I admit it's astonishingly complex and capable. So are honeybees. But when bees perform unbelievably complicated tasks, I don't assume they therefore have human-level intelligence, and I think the unconscious' actions are more like the honeybees' than people's.

However, if there's something you think I should know more about, why not recommend me specific articles, authors, or books?

Comment author: SilasBarta 12 August 2009 04:30:06AM 3 points [-]

irrationality of unconscious mind continuing to pursue subgoals when clearly no longer connected to supergoals, unconscious' vulnerability to proximity/scale biases when dealing with morality, and several others.

The conscious is guilty of these too.

I've read plenty about the unconscious, and I admit it's astonishingly complex and capable. So are honeybees. But when bees perform unbelievably complicated tasks, I don't assume they therefore have human-level intelligence, and I think the unconscious' actions are more like the honeybees' than people's.

Okay, but carrying the analogy over, I'm sure you also don't trivialize the value of honey!

However, if there's something you think I should know more about, why not recommend me specific articles, authors, or books?

You could start with making yourself aware of the non-conscious mind's ability to solve CAPTCHAs, an AI-complete problem, and current conscious minds' inability to figure out how they do it with enough clarity to re-implement it in software.

Comment author: Yvain 17 August 2009 04:57:59PM *  4 points [-]

Actually, it's funny you mention CAPTCHAs as your example. If you're going to go that far, why not also attribute skill at chess to the unconscious? After all, it's got to be the unconscious that screens out most of the several dozen possible chess moves each turn and lets your conscious concentrate on the few best, and you can generalize from chess to practically all matters of strategy. Or for that matter, how about language? All my knowledge of English grammar was purely unconscious until I started studying the subject in high school, and 99% of my grammar use still comes from there.

So the issue's not whether it can perform complex tasks. I don't know exactly what the issue is, but I think it connects to the concept of "personhood" somehow. I question whether the unconscious is more than a collection of very sophisticated mental modules, in the same way that a bird's brain may have a flight dynamics module, an astronomical navigation module, a mate-preference-analysis module, and so on.

The computing hardware of my brain contains a program for recognizing letters, a program that detects potential mates and responds with feelings of lust, a program that interacts with my reward system in such a way as to potentially create alcoholism, and so on. They're all computationally very impressive. But I don't see why I should assign them moral status any more than I would feel morally obligated to listen to a laptop on which I had installed a program that detected the presence of beautiful women nearby and then displayed the words "mate with this woman". I don't want to privilege these programs just because they happen to be located inside a human brain and they get reflected glory from some of the other things human brains can do.

To make me want to assign them moral status, you'd have to give me evidence that there was something that it felt like to be my lust. This seems kind of category-error-ish to me. I feel my lust, but my lust itself doesn't feel anything. You may feel sorry for me for having to deal with my lust, but feeling sorry for my lust because I don't choose to satisfy it is in my opinion a waste of sorrow. It's also an infinite regress. If I feel unhappy because I have unfulfilled desire, and my desire feels unhappy because it's unfulfilled, does my desire's unhappiness feel something? Why stop there?

I have a feeling this problem requires more rigor than I can throw at it right now. I've been trying to think about it more clearly so as to hopefully eventually get some top-level posts out of it, but this is the best I can do at the moment.

Comment author: Eliezer_Yudkowsky 17 August 2009 06:11:25PM 3 points [-]

I question whether the unconscious is more than a collection of very sophisticated mental modules

So's your conscious. The unconscious just isn't connected up the right way for deliberation and reflectivity.

(IAWYC)

Comment author: SilasBarta 19 August 2009 03:34:28PM 0 points [-]

I'll bite the bullets in your first paragraph. So chess also relies on non-conscious skills. What trap did I just fall into?

I don't see why I should assign them moral status any more than I would feel morally obligated to listen to a laptop ...

There is a major difference between your unconscious mind and a laptop with the same output: specifically, the unconscious mind has a direct, seamless, high-bandwidth connection to your mind. When you recognize a face or a letter, you don't have to pass it to a laptop, look at the output, and read the output. From your conscious mind's perspective, you just get insta-recognition. This makes it more valuable that a laptop -- in all senses -- just as faster mental addition is better than a hand calculator that computes with the same speed.

If and when someone makes a machine that can do these tasks faster, and still interface seamlessly, in the unconscious's stead, then you will be justified in trivializing the latter's value. Just like you would feel less bad (though not completely indifferent) about the extinction of honeybees if honey could be more efficiently sythesized.

The only case where the above reasoning doens't apply is, as you point out, in values. Why is the unconscious mind's decision of values, er, valuable? Why are you morally bound to its decrees of lust? There answer is, I don't know. But at the same time, I don't know how you can clip out the lust while retaining "you" -- not given your existing brain's architecture. That is, I disagree that the brain is as modular as you seem to think, at least if that's what you meant by the use of "modules".

And remember, pure value judgments are only a small fraction of its outputs.

Comment author: timtyler 18 August 2009 07:30:06PM 0 points [-]

Re: I question whether the unconscious is more than a collection of very sophisticated mental modules, in the same way that a bird's brain may have a flight dynamics module, an astronomical navigation module, a mate-preference-analysis module, and so on.

...and what do you think your conscious mind is, then - if not a collection of sophisticated mental modules?

Comment author: Yvain 18 August 2009 09:51:58PM *  0 points [-]

Wikipedia gives Fodor's list of eight characteristics of "mental modules", which include "domain specificity", "fast speed", "shallow output", "limited accessibility", "encapsulation", et cetera, and quotes someone else as saying the most important distinguishing feature is "cognitive impenetrability".

In other words, "module" has a special definition that doesn't mean exactly the same as "something in the mind". So when I "accuse" the unconscious of being "modules", all I'm saying is that it's a bunch of single-purpose unlinked programs, as opposed to the generic and unified programs that make up the conscious mind. This seems relevant since it makes it harder to accept the idea of the unconscious as a separate but equal person living inside your brain.

If there are other definitions of "module" that include anything in the mind, and you're using one of those, then yes, the conscious mind is a module or collection of modules as well.

Comment author: RobinHanson 17 August 2009 01:14:21PM 1 point [-]

Just because your conscious mind isn't aware of experiences by your unconscious mind doesn't mean they don't exit. And just because your unconscious is subject to some biases doesn't mean your conscious mind does better on average.

Comment author: Yvain 17 August 2009 04:40:33PM 2 points [-]

I don't disagree with any of that, but it's all phrased sort of as "you can't prove it doesn't." What should make me single out the hypothesis that it does as worthy of further consideration?

Comment author: RobinHanson 18 August 2009 06:32:22PM 2 points [-]

I don't know how else to say it: the things you point to as evidence supporting your claims just don't actually offer substantial support for those claims. To support claims about the relative features of two systems you need relative evidence; absolute evidence about one system just isn't very relevant.

Comment author: teageegeepea 12 August 2009 04:52:20AM 1 point [-]

Maybe individual honeybees aren't very intelligent, but hives are moreso. Hopefully Anonymous often makes a similar point about markets, corporations or other collective entities and suggests some might even be (or become) . I don't really care much about consciousness, but viewed as persisting and replicating entities they might be lumped in with other life (just like multicellular and unicellular life are).

Comment author: timtyler 18 August 2009 07:27:59PM -2 points [-]

The conscious mind does some pretty stupid things too. Like becoming a catholic priest. That sort of thing consigns your potentially-immortal essence that's responsible for your very exisence to the trash bin.

If this is a battle to see which system is the more stupid, we could be looking at examples of insanity from both sides all day.

Comment author: arundelo 10 August 2009 10:07:31PM 0 points [-]

Any idea what opinions your unconscious mind has on these matters?

If it had any, do you think it would be incapable of letting us know about them? If so, why?

(Maybe Yvain should have a session with a Ouija board.)

Comment author: timtyler 18 August 2009 07:21:34PM -2 points [-]

Your definitions of "conscious" and "unconscious" seem highly irregular to me. Best to stick to the dictionary here - I figure.

Comment author: christopherj 25 January 2014 07:38:22AM 0 points [-]

My beliefs about the rationality and morality of my unconscious mind do come from my conscious mind, this is where all my beliefs come from.

I'd dispute this. Just as an example, it is your unconscious mind which provides the processing power for you to read this statement, to understand spoken words, etc. It is largely your unconscious mind which declared some words interesting and others boring. It is your unconscious mind which declares things pleasant (and therefore morally good if you value hedonism). It is your unconscious that contains mirror neurons, without which your morality might be rather different. It is your unconscious that remembers and forgets, though with repeated effort and a consequent use of willpower your conscious can demand a few specific facts be remembered. Your conscious mind may have devised your moral system, but where did the initial values you seek to maximize come from?

I have to either forfeit my right to judge entirely, or use those same judgments to judge my unconscious.

Does it do you any good to judge your unconscious? If you could accomplish more of your conscious goals if you had more willpower, perhaps you could accomplish more of your conscious goals if you found a way to spend less willpower fighting your unconscious.