Eliezer_Yudkowsky comments on The Twin Webs of Knowledge - Less Wrong

5 Post author: Kaj_Sotala 28 August 2009 09:45AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (72)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 28 August 2009 07:31:43PM *  11 points [-]

Finding out about human nature, and particularly your own nature is typically a soul-destroying experience,

Either we have very different natures or you're doing it wrong. Are you sure you're really understanding your own nature, or are you just being told in a flat voice that it's non-mysterious but without actually having the sort of knowledge you'd need to e.g. build a copy of yourself?

I've been pleasantly surprised by the fact that every piece of knowledge I feared has turned out not to cause an internal catastrophe once I obtained it.

Comment author: SforSingularity 28 August 2009 07:52:49PM *  3 points [-]

I wouldn't say "internal catastrophe", more just a long line of disappointments. To wit: finding out that when we humans profess our undying love for each other, we are actually simply deluded. The real nature of human relationships is a compromise between co-operation and defection, as evidence by human nonpaternity studies.

Or, take the example of charity and the extent to which many people give just enough money to charity to purchase moral satisfaction and no more. And the degree to which people are eager to help people like tramps who are near and immediate, but not the much more worthy cause of third world poverty. And also the extent to which people react badly to suggestions about efficient charity.

Hell, consider that most people just don't give anything to charity, and don't think that there's a problem with ordering their nice new flatscreen TV whilst the kids in Africa die of malaria. And, of course, there's an evo-psych explanation for this.

Or, to take a leaf from the book of Frank Adamek, consider the extent to which humans do not magically become super-motivated and super effective when they know that their actions determine, with non-negligible probability, the fate of the universe:

What we do have are foibles, eccentricities, and fixations. We have imperfections and disabilities, irrational modes of thought and poor calibration. We’re dragged down by fear and self-doubt and insecurities. We’re given to rash and ineffective violence, and to thinking in tribalistic, us-versus-them mindsets. We shake and we cry and we bleed, we get sick and we get disparaged and we get depressed.

In each of these cases, more self knowledge shatters our pleasant delusions about ourselves.

Now, since I am still here, I haven't had an "internal catastrophe" upon learning these things, because as PJEby says, knowing your weaknesses is the first step to overcoming them. In essence, this little comment explains both the urgent need for transhumanism, and why it is so unpopular.

And lastly, the best thing one could ever learn about human nature is that we will succeed this century *in spite of * our flaws. And that, I guess, is singularitarianism. Unfortunately, it is a dream which may or may not come true.

Comment author: kess3r 29 August 2009 12:29:16AM *  1 point [-]
Comment author: Psychohistorian 29 August 2009 12:11:17AM 1 point [-]

I'm not sure where you got your standards, but I'm reasonably sure they're a bit high.

Comment author: SforSingularity 29 August 2009 10:23:07AM *  0 points [-]

Irrelevant - the claim is whether or not more self-knowledge is a happy set of surprises or a set of mostly disappointments. I do not think that my pre rationaliy expectations about human nature were unusual for a human being. Maybe this is your claim? You think that there are people who read evolutionary psychology and were pleasantly surprised?

Comment author: Kaj_Sotala 29 August 2009 05:38:50PM 4 points [-]

You think that there are people who read evolutionary psychology and were pleasantly surprised?

I've found a definite relief in evolutionary psychology - as many others, I have maintained an unrealistically positive self-image. Then at times I have found out that my actions don't match up with the ethics I was previously claiming to follow. Looking at evpsych and realizing that this kind of behavior is actually normal has helped me to accept that I don't need to feel guilty about being less ethical than I actually am... and accepting that has helped me actually become more ethical, in more ways than one, as I don't need to waste time feeling guilty instead of actually changing things.

Comment author: SforSingularity 29 August 2009 05:53:59PM *  2 points [-]

Looking at evpsych and realizing that this kind of behavior is actually normal has helped me to accept that I don't need to feel guilty about being less ethical than I actually am

Now that is an interesting take on the matter. Thank you, Kaj.

Of course, before the "realization" that your misdemeanours were caused by lawful physical malfunctions of your brain, rather than by a nonphysical black box called your "self", one could always entertain the illusion that misbehaviour was, for oneself, an abberation which would be expunged if only you really tried hard enough. To realize that it is the default scenario is saddening.

Comment author: Jonathan_Graehl 01 September 2009 01:12:00AM 0 points [-]

I don't need to feel guilty about being less ethical than I actually am

clarify: about the possibility of being less ethical than you are now? Obviously you can't now be less than you are now.

Comment author: Kaj_Sotala 01 September 2009 05:57:49PM 1 point [-]

I think I meant to write something along the lines of "about acting less ethically than my unrealistically glorified self-image claims I would act".

Comment author: scotherns 31 August 2009 02:27:45PM 3 points [-]

You think that there are people who read evolutionary psychology and were pleasantly surprised?

I was VERY pleasantly surprised. Suddenly an enormous set of previously baffling data (i.e. the behaviour of most of humanity) began to make sense :-)

It's hard to fix the root cause of a problem without understanding it.

Comment author: SforSingularity 31 August 2009 04:18:03PM *  0 points [-]

It's hard to fix the root cause of a problem without understanding it.

If I had simultaneously discovered Evo Psych, and a viable strategy to debias the human race quickly, I would share your enthusiasm... as it is, the situation could be construed as hopeless, so it might be better if we lived out our lives in ignorance. Whether it is actually hopeless is another question, and one that I want to answer.

Comment author: scotherns 01 September 2009 07:57:09AM 0 points [-]

I thought it was hopeless before I discovered Evo Psych. Now it's just very difficult.

Quickly debiasing the human race seems a bit optimistic :-) Knowing Evo Psych at least makes it possible to make better predictions, and take more effective action. How can this be a bad thing?

Comment author: Douglas_Knight 30 August 2009 06:17:57AM 2 points [-]

I think your expectations about human nature were unusual, though typical for a nerd. They're probably what everyone verbalizes, but you're a nerd who is dominated by words, rather than paying attention to (and imitating) how people actually act. I think the answer to Psychohistorian's question about where you got your standards is from other people, who described them in the language of Normal people, while you spoke only Nerd.

Also, your complaints are all phrased in terms of other people, not self-knowledge. It is compatible with your claims that you live up to your standards and other people just don't hold them. In particular, you complain that you're not important because people don't act. But if most people don't act, there's little competition to be important! That doesn't mean it's easy, but it means that it's difficult in ways that are different than you thought before, and you have the advantage of knowing this.

Probably you don't live up to your standards, but pay attention and check what you actually do. Don't take ev psych's word for it, since (I claim) you got in this mess by paying too much attention to words.

Comment author: SforSingularity 30 August 2009 07:09:28PM *  0 points [-]

They're probably what everyone verbalizes, but you're a nerd who is dominated by words, rather than paying attention to (and imitating) how people actually act. I think the answer to Psychohistorian's question about where you got your standards is from other people, who described them in the language of Normal people, while you spoke only Nerd.

So basically, the solution to the problem of being depressed because I now have too much knowledge about my own, and others' flaws is to get one more piece of knowledge: nobody else really believes in these standards, and furthermore are are continually emitting Genuine BullShit (tm) when they speak about standards - i.e. they compartmentalize - abstract ethics goes in one compartment, actual criteria for taking actions go in another.

Comment author: Douglas_Knight 30 August 2009 08:13:57PM 1 point [-]

So basically, the solution to the problem of being depressed because I now have too much knowledge

Maybe my comment mislead because of the context. I didn't say it was a solution. Mainly, I meant to unbundle "what" from "why." I think it is what people do that bothers you. For people who are already disappointed by "what," learning "why" might be a positive experience.

I didn't say that self-knowledge makes you happy, though I agree with Kaj Sotala. And self-knowledge is necessary for self-improvement, for you produce your own happiness.

Comment author: SforSingularity 30 August 2009 08:28:50PM *  1 point [-]

Mainly, I meant to unbundle "what" from "why." I think it is what people do that bothers you. For people who are already disappointed by "what," learning "why" might be a positive experience.

I think that people who do not know about human cognitive biases tend to hold lots of false beliefs on the "what" side, for example by employing various pieces of dark side epistemology to protect certain cherished false beliefs about human nature.

And self-knowledge is necessary for self-improvement, for you produce your own happiness.

yes but self-knowledge is not necessary for happiness - let us be clear, you might never get as much happiness back through effort as you lost through debiasing. Not that that bothers me, because I value truth very highly, but it would bother some people.

Comment author: Psychohistorian 30 August 2009 08:47:56PM 0 points [-]

Quite simply, there is nothing inherently "depressing" or "disappointing" with how people happen to be. It would be nice if people were genuinely charitable, and, to the degree that it's intelligible, it would be nice if love were more than "mere chemical reactions." But it's never been this way, and neither will ever likely actually happen. The fundamental problem is that your reaction works as if changing your understanding changed the world, rather than the other way around.

What I meant by high standards specifically is that one need not think people are perfectly charitable to generally like people. People you don't know behaving somewhat worse than you would hope is not a reason to become dispirited, particularly when they were never that way to begin with.

Comment author: SforSingularity 30 August 2009 09:01:00PM *  0 points [-]

Quite simply, there is nothing inherently "depressing" or "disappointing" with how people happen to be

"Depressing" is a 2-place predicate - Depressing(x,y). A certain situation x may or may not be depressing to a certain individual y. The situation that humans are both uncharitable, selfish and furthermore deluded about that is depressing to me.

Causally, this is because I also used to be deluded about it, so finding out that people are not as nice as the propoganda says they are feels like a loss, though, as you point out, it is not.

But the fact that the causal explanation for my disappointment in humanity is that I used to be deluded does not logically compel me to change my standards.

Indeed, I think that it is precisely because we are mostly deluded about what our own typical behaviour is, and what our typical motivations are that we even have a concept of goodness. Our concept of goodness is what happens when we believe our own bullshit.

Comment author: Vladimir_Nesov 28 August 2009 08:14:44PM *  -2 points [-]

And lastly, the best thing one could ever learn about human nature is that we will succeed this century *in spite of * our flaws. And that, I guess, is singularitarianism. Unfortunately, it is a dream which may or may not come true.

Huh? Some teleology. Obligatory reading: Existential risk.

Comment author: SforSingularity 29 August 2009 10:19:20AM 1 point [-]

I said may or may not come true - I realize that existential risk is the main power behind "may not"

Comment author: Vladimir_Nesov 29 August 2009 10:32:25AM -1 points [-]

You did say that, but you also said that "we will succeed this century *in spite of* our flaws", which seems like a clear contradiction.

Comment author: [deleted] 29 August 2009 09:41:30AM 1 point [-]

Suppose that you're just an ordinary person, with an ordinary person's personality, habits, and idiosyncrasies. Then, somehow, you come to find out that your personality, habits, and idiosyncrasies are all caused by the fact that there's not enough iron in your blood, or some of the proteins in your brain are misbehaving, and you fix the problem, causing your personality, habits, and idiosyncrasies to be erased and replaced with something else. Maybe you're happier and also more skilled as a result. But what part of you has been preserved?

Comment author: CannibalSmith 31 August 2009 11:20:23AM 0 points [-]

My feeling of existence.

Comment author: [deleted] 31 August 2009 04:17:17PM 0 points [-]

What if I just replaced you with a copy of, I dunno, my high school English teacher? You would still have a feeling of existence if I did that.

Comment author: CannibalSmith 31 August 2009 08:09:21PM 0 points [-]

If you did that gradually, maybe.

Comment author: Douglas_Knight 30 August 2009 03:06:19AM 0 points [-]

I've been pleasantly surprised by the fact that every piece of knowledge I feared has turned out not to cause an internal catastrophe once I obtained it.

What did you fear?

Maybe I'm reading too much into this phrasing, but I really don't like that "once I obtained it." It would be plausible if you were saying that you feared things from afar, but once you were forced to deal with them, it turned out that you could live with them, and you could have figured that out if you'd confronted them ahead of time. But it sounds like you're saying that they only turned out to be unproblematic when you got close to them, which sounds suspicious to me.