SforSingularity comments on The Twin Webs of Knowledge - Less Wrong

5 Post author: Kaj_Sotala 28 August 2009 09:45AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (72)

You are viewing a single comment's thread.

Comment author: SforSingularity 28 August 2009 05:30:47PM *  1 point [-]

There are those among us who resist the steady march of science, who feel that the reductionist creed takes away the beauty of things, who would rather enjoy sacred mysteries instead of naturalistic explanations. I suspect not many of them are reading this site. But even among aspiring rationalists, there are probably many who still feel some sympathy to that line of thought, who cannot but feel a twinge of pain where something mysterious ends up explained.

I can make a stronger complaint than that: that the more we find out about ourselves, the emptier we feel. Finding out about other things is usually a pleasant and exciting experience. Finding out about human nature, and particularly your own nature is typically a soul-destroying experience, because we all started off with an inflated sense of our own value, importance, wisdom, etc. We also have a strong intuitive belief in our own un-understandability - that we are "free agents" independent from the laws of the universe. This, it turns out, is false.

Comment author: Eliezer_Yudkowsky 28 August 2009 07:31:43PM *  11 points [-]

Finding out about human nature, and particularly your own nature is typically a soul-destroying experience,

Either we have very different natures or you're doing it wrong. Are you sure you're really understanding your own nature, or are you just being told in a flat voice that it's non-mysterious but without actually having the sort of knowledge you'd need to e.g. build a copy of yourself?

I've been pleasantly surprised by the fact that every piece of knowledge I feared has turned out not to cause an internal catastrophe once I obtained it.

Comment author: SforSingularity 28 August 2009 07:52:49PM *  3 points [-]

I wouldn't say "internal catastrophe", more just a long line of disappointments. To wit: finding out that when we humans profess our undying love for each other, we are actually simply deluded. The real nature of human relationships is a compromise between co-operation and defection, as evidence by human nonpaternity studies.

Or, take the example of charity and the extent to which many people give just enough money to charity to purchase moral satisfaction and no more. And the degree to which people are eager to help people like tramps who are near and immediate, but not the much more worthy cause of third world poverty. And also the extent to which people react badly to suggestions about efficient charity.

Hell, consider that most people just don't give anything to charity, and don't think that there's a problem with ordering their nice new flatscreen TV whilst the kids in Africa die of malaria. And, of course, there's an evo-psych explanation for this.

Or, to take a leaf from the book of Frank Adamek, consider the extent to which humans do not magically become super-motivated and super effective when they know that their actions determine, with non-negligible probability, the fate of the universe:

What we do have are foibles, eccentricities, and fixations. We have imperfections and disabilities, irrational modes of thought and poor calibration. We’re dragged down by fear and self-doubt and insecurities. We’re given to rash and ineffective violence, and to thinking in tribalistic, us-versus-them mindsets. We shake and we cry and we bleed, we get sick and we get disparaged and we get depressed.

In each of these cases, more self knowledge shatters our pleasant delusions about ourselves.

Now, since I am still here, I haven't had an "internal catastrophe" upon learning these things, because as PJEby says, knowing your weaknesses is the first step to overcoming them. In essence, this little comment explains both the urgent need for transhumanism, and why it is so unpopular.

And lastly, the best thing one could ever learn about human nature is that we will succeed this century *in spite of * our flaws. And that, I guess, is singularitarianism. Unfortunately, it is a dream which may or may not come true.

Comment author: kess3r 29 August 2009 12:29:16AM *  1 point [-]
Comment author: Psychohistorian 29 August 2009 12:11:17AM 1 point [-]

I'm not sure where you got your standards, but I'm reasonably sure they're a bit high.

Comment author: SforSingularity 29 August 2009 10:23:07AM *  0 points [-]

Irrelevant - the claim is whether or not more self-knowledge is a happy set of surprises or a set of mostly disappointments. I do not think that my pre rationaliy expectations about human nature were unusual for a human being. Maybe this is your claim? You think that there are people who read evolutionary psychology and were pleasantly surprised?

Comment author: Kaj_Sotala 29 August 2009 05:38:50PM 4 points [-]

You think that there are people who read evolutionary psychology and were pleasantly surprised?

I've found a definite relief in evolutionary psychology - as many others, I have maintained an unrealistically positive self-image. Then at times I have found out that my actions don't match up with the ethics I was previously claiming to follow. Looking at evpsych and realizing that this kind of behavior is actually normal has helped me to accept that I don't need to feel guilty about being less ethical than I actually am... and accepting that has helped me actually become more ethical, in more ways than one, as I don't need to waste time feeling guilty instead of actually changing things.

Comment author: SforSingularity 29 August 2009 05:53:59PM *  2 points [-]

Looking at evpsych and realizing that this kind of behavior is actually normal has helped me to accept that I don't need to feel guilty about being less ethical than I actually am

Now that is an interesting take on the matter. Thank you, Kaj.

Of course, before the "realization" that your misdemeanours were caused by lawful physical malfunctions of your brain, rather than by a nonphysical black box called your "self", one could always entertain the illusion that misbehaviour was, for oneself, an abberation which would be expunged if only you really tried hard enough. To realize that it is the default scenario is saddening.

Comment author: Jonathan_Graehl 01 September 2009 01:12:00AM 0 points [-]

I don't need to feel guilty about being less ethical than I actually am

clarify: about the possibility of being less ethical than you are now? Obviously you can't now be less than you are now.

Comment author: Kaj_Sotala 01 September 2009 05:57:49PM 1 point [-]

I think I meant to write something along the lines of "about acting less ethically than my unrealistically glorified self-image claims I would act".

Comment author: scotherns 31 August 2009 02:27:45PM 3 points [-]

You think that there are people who read evolutionary psychology and were pleasantly surprised?

I was VERY pleasantly surprised. Suddenly an enormous set of previously baffling data (i.e. the behaviour of most of humanity) began to make sense :-)

It's hard to fix the root cause of a problem without understanding it.

Comment author: SforSingularity 31 August 2009 04:18:03PM *  0 points [-]

It's hard to fix the root cause of a problem without understanding it.

If I had simultaneously discovered Evo Psych, and a viable strategy to debias the human race quickly, I would share your enthusiasm... as it is, the situation could be construed as hopeless, so it might be better if we lived out our lives in ignorance. Whether it is actually hopeless is another question, and one that I want to answer.

Comment author: scotherns 01 September 2009 07:57:09AM 0 points [-]

I thought it was hopeless before I discovered Evo Psych. Now it's just very difficult.

Quickly debiasing the human race seems a bit optimistic :-) Knowing Evo Psych at least makes it possible to make better predictions, and take more effective action. How can this be a bad thing?

Comment author: Douglas_Knight 30 August 2009 06:17:57AM 2 points [-]

I think your expectations about human nature were unusual, though typical for a nerd. They're probably what everyone verbalizes, but you're a nerd who is dominated by words, rather than paying attention to (and imitating) how people actually act. I think the answer to Psychohistorian's question about where you got your standards is from other people, who described them in the language of Normal people, while you spoke only Nerd.

Also, your complaints are all phrased in terms of other people, not self-knowledge. It is compatible with your claims that you live up to your standards and other people just don't hold them. In particular, you complain that you're not important because people don't act. But if most people don't act, there's little competition to be important! That doesn't mean it's easy, but it means that it's difficult in ways that are different than you thought before, and you have the advantage of knowing this.

Probably you don't live up to your standards, but pay attention and check what you actually do. Don't take ev psych's word for it, since (I claim) you got in this mess by paying too much attention to words.

Comment author: SforSingularity 30 August 2009 07:09:28PM *  0 points [-]

They're probably what everyone verbalizes, but you're a nerd who is dominated by words, rather than paying attention to (and imitating) how people actually act. I think the answer to Psychohistorian's question about where you got your standards is from other people, who described them in the language of Normal people, while you spoke only Nerd.

So basically, the solution to the problem of being depressed because I now have too much knowledge about my own, and others' flaws is to get one more piece of knowledge: nobody else really believes in these standards, and furthermore are are continually emitting Genuine BullShit (tm) when they speak about standards - i.e. they compartmentalize - abstract ethics goes in one compartment, actual criteria for taking actions go in another.

Comment author: Douglas_Knight 30 August 2009 08:13:57PM 1 point [-]

So basically, the solution to the problem of being depressed because I now have too much knowledge

Maybe my comment mislead because of the context. I didn't say it was a solution. Mainly, I meant to unbundle "what" from "why." I think it is what people do that bothers you. For people who are already disappointed by "what," learning "why" might be a positive experience.

I didn't say that self-knowledge makes you happy, though I agree with Kaj Sotala. And self-knowledge is necessary for self-improvement, for you produce your own happiness.

Comment author: SforSingularity 30 August 2009 08:28:50PM *  1 point [-]

Mainly, I meant to unbundle "what" from "why." I think it is what people do that bothers you. For people who are already disappointed by "what," learning "why" might be a positive experience.

I think that people who do not know about human cognitive biases tend to hold lots of false beliefs on the "what" side, for example by employing various pieces of dark side epistemology to protect certain cherished false beliefs about human nature.

And self-knowledge is necessary for self-improvement, for you produce your own happiness.

yes but self-knowledge is not necessary for happiness - let us be clear, you might never get as much happiness back through effort as you lost through debiasing. Not that that bothers me, because I value truth very highly, but it would bother some people.

Comment author: Psychohistorian 30 August 2009 08:47:56PM 0 points [-]

Quite simply, there is nothing inherently "depressing" or "disappointing" with how people happen to be. It would be nice if people were genuinely charitable, and, to the degree that it's intelligible, it would be nice if love were more than "mere chemical reactions." But it's never been this way, and neither will ever likely actually happen. The fundamental problem is that your reaction works as if changing your understanding changed the world, rather than the other way around.

What I meant by high standards specifically is that one need not think people are perfectly charitable to generally like people. People you don't know behaving somewhat worse than you would hope is not a reason to become dispirited, particularly when they were never that way to begin with.

Comment author: SforSingularity 30 August 2009 09:01:00PM *  0 points [-]

Quite simply, there is nothing inherently "depressing" or "disappointing" with how people happen to be

"Depressing" is a 2-place predicate - Depressing(x,y). A certain situation x may or may not be depressing to a certain individual y. The situation that humans are both uncharitable, selfish and furthermore deluded about that is depressing to me.

Causally, this is because I also used to be deluded about it, so finding out that people are not as nice as the propoganda says they are feels like a loss, though, as you point out, it is not.

But the fact that the causal explanation for my disappointment in humanity is that I used to be deluded does not logically compel me to change my standards.

Indeed, I think that it is precisely because we are mostly deluded about what our own typical behaviour is, and what our typical motivations are that we even have a concept of goodness. Our concept of goodness is what happens when we believe our own bullshit.

Comment author: Vladimir_Nesov 28 August 2009 08:14:44PM *  -2 points [-]

And lastly, the best thing one could ever learn about human nature is that we will succeed this century *in spite of * our flaws. And that, I guess, is singularitarianism. Unfortunately, it is a dream which may or may not come true.

Huh? Some teleology. Obligatory reading: Existential risk.

Comment author: SforSingularity 29 August 2009 10:19:20AM 1 point [-]

I said may or may not come true - I realize that existential risk is the main power behind "may not"

Comment author: Vladimir_Nesov 29 August 2009 10:32:25AM -1 points [-]

You did say that, but you also said that "we will succeed this century *in spite of* our flaws", which seems like a clear contradiction.

Comment author: [deleted] 29 August 2009 09:41:30AM 1 point [-]

Suppose that you're just an ordinary person, with an ordinary person's personality, habits, and idiosyncrasies. Then, somehow, you come to find out that your personality, habits, and idiosyncrasies are all caused by the fact that there's not enough iron in your blood, or some of the proteins in your brain are misbehaving, and you fix the problem, causing your personality, habits, and idiosyncrasies to be erased and replaced with something else. Maybe you're happier and also more skilled as a result. But what part of you has been preserved?

Comment author: CannibalSmith 31 August 2009 11:20:23AM 0 points [-]

My feeling of existence.

Comment author: [deleted] 31 August 2009 04:17:17PM 0 points [-]

What if I just replaced you with a copy of, I dunno, my high school English teacher? You would still have a feeling of existence if I did that.

Comment author: CannibalSmith 31 August 2009 08:09:21PM 0 points [-]

If you did that gradually, maybe.

Comment author: Douglas_Knight 30 August 2009 03:06:19AM 0 points [-]

I've been pleasantly surprised by the fact that every piece of knowledge I feared has turned out not to cause an internal catastrophe once I obtained it.

What did you fear?

Maybe I'm reading too much into this phrasing, but I really don't like that "once I obtained it." It would be plausible if you were saying that you feared things from afar, but once you were forced to deal with them, it turned out that you could live with them, and you could have figured that out if you'd confronted them ahead of time. But it sounds like you're saying that they only turned out to be unproblematic when you got close to them, which sounds suspicious to me.

Comment author: pjeby 28 August 2009 06:35:46PM 2 points [-]

Finding out about human nature, and particularly your own nature is typically a soul-destroying experience, because we all started off with an inflated sense of our own value, importance, wisdom, etc.

That's only true if the insight isn't accompanied by the ability to change that nature. Finding out just how messed up you are is liberating when you realize that 1) you're not to blame for the past, and 2) it doesn't have to be your future any more.

Comment author: Vladimir_Nesov 28 August 2009 06:49:55PM 0 points [-]

There is no ability to change human nature as yet.

Comment author: thomblake 28 August 2009 07:43:44PM 3 points [-]

What do you mean by 'human nature'? It seems like 'nature' is in general ill-defined in the first place. If you mean simply, "what it's like to be a human" then I think in many relevant ways we're changing that all the time.

Comment author: gwern 29 August 2009 01:45:37PM 0 points [-]

As the Nietzschean quip goes, if the human nature is to be in flux, then you can't change it by changing yourself or others.

Comment author: PhilGoetz 29 August 2009 06:47:32PM 1 point [-]

You can change it in 5 minutes with an icepick.

Comment author: ferrouswheel 28 August 2009 11:09:37PM 0 points [-]

Modafinil removes the urge to sleep pretty well - but as thomblake mentions, it depends on how you define that ill-defined concept.

Comment author: pjeby 29 August 2009 05:16:56PM 0 points [-]

There is no ability to change human nature as yet.

If you re-read the comment you're replying to, you'll see I was answering the part about "your own nature", not "human nature".

However, if you consider that most of what constitutes "human nature" is actually metaprogramming that drives the acquisition of our individual nature, then an enormous part of that nature is not actually hard-wired.

People who've not done any significant amount of mindhacking are horribly biased towards believing that aspects of their individual nature are in fact universal. (Actually, everyone is so biased, it's just that non-mindhackers are an order of magnitude worse, because they don't have the experience yet of seeing the consistent disconnect between their automatic thoughts and external reality.)

Stupid example: earlier this week I realized that I was choosing not to aggressively pursue certain goals because I felt the "rush" to complete them would be stressful. Then I realized that there was no intrinsic association between "rush" and "stress" -- that was a learned response, and a fairly specific one at that. (My mother always freaked out whenever she was late... which was virtually all the time.)

However, until I thought to question that specific assumption, I was not conscious of it being my individual nature - it was assumed to be part of human nature, or just the nature of the world itself. (i.e. "of course it's stressful to rush")

It's impractical to question every implicit association, though, and practical knowledge/experience is needed as a guide to know what assumptions to surface and question. (A good rule of thumb, though, is that anything that provokes a negative emotional reaction should be questioned thoroughly.)

Comment author: wedrifid 28 August 2009 07:32:49PM 0 points [-]

For the individual at least. Eugenics of course gives the ability to change human nature (albeit with a time scale and logistical difficulties that make it useless to most purposes).

Comment author: SforSingularity 28 August 2009 07:20:38PM 0 points [-]

This is true, to some extent.

We are able to make small changes to human nature, including our own nature. Just how far do you think that can be taken, though? Can a lazy person recognize that they are lazy and go on to become a millionaire?

Comment author: pjeby 29 August 2009 05:04:01PM 0 points [-]

Just how far do you think that can be taken, though? Can a lazy person recognize that they are lazy and go on to become a millionaire?

I'll wait to answer that question until I become a millionaire a second time, so I can replicate the result. ;-) (The first time might have been overly influenced by the dotcom boom, and in any event didn't involve me changing any personal characteristics.)

More seriously: I've changed a ton of other characteristics in myself, both minor and major, and this is not uncommon among people I've trained. Realizing that you do something for a reason -- even if it's a stupid, outdated, reason -- is often a relief in itself. But once you understand the reason, then using the right method(s) allows change to take place almost instantaneously. It's finding the reasons in the first place that's more complex, since the brain does not (alas) have a "view source" button.

Comment author: SforSingularity 29 August 2009 05:14:32PM *  0 points [-]

How did you first become a millionaire?

Comment author: pjeby 29 August 2009 05:21:28PM 0 points [-]

How did you first become a millionaire?

Stock options.

Comment author: SforSingularity 29 August 2009 05:48:39PM 0 points [-]

In a dotcom startup that made it big? Were you a founder? Early employee?

Comment author: pjeby 29 August 2009 06:12:54PM 1 point [-]

In a dotcom startup that made it big? Were you a founder? Early employee?

Something like that. I designed a customer ticket tracking system that decreased customer-perceived latency of cases handled by email (using a novel scheduling technique derived from the Theory of Constraints), improved interdisciplinary co-operation across a physically and organizationally-distributed workforce, and enabled private-label support for OEM clients without needing dedicated staff for each client.

It made a huge difference to the ability to land private-label deals while keeping the overhead for each deal low, as well as easing product-line integration as the company expanded. In terms of value to the company, it was probably worth at least $20-30million during the time I worked there. I probably should've asked for more options. ;-)

Comment author: SforSingularity 29 August 2009 06:44:16PM 0 points [-]

And the skills required for this were? Programming experience and your innate intelligence, plus a modicum of business sense and what we would call rationality here?

Comment author: pjeby 29 August 2009 10:20:08PM 1 point [-]

And the skills required for this were? Programming experience and your innate intelligence, plus a modicum of business sense and what we would call rationality here?

I don't think that much of what gets discussed here would've helped much or been particularly relevant. It was more a matter of knowing what I wanted and what the company needed, as well as my previous 12+ years practical experience about what people will and won't do when confronted with a computer program that they don't necessarily want to use in the first place.

It was a problem of whole-system design, including both social and HCI aspects. For example, many characteristics of the system were designed specifically to promote viral adoption of the software within the company, as well as to create subtle social pressures towards customer- or company-beneficial behaviors. I had previously apprenticed under a teacher who taught me the social dynamics of business, as well as the art of designing systems that merged machine and human information processing with social manipulation to achieve business goals. (Specifically, in the field of real estate office management software, but the lessons were pretty universal.)

In a way, you could say my understanding of irrationality was at least as important, if not far more important, than my understanding of "rationality". (Though I learned a lot of instrumental rationality principles during my apprenticeship as well.)

Comment author: PhilGoetz 29 August 2009 06:43:22PM *  0 points [-]

I can think of 6 of my friends who became multi-millionaires at a young age. 4 of them got wild lucky breaks with stock options; 1 of them inherited it; 1 of them worked for 10 years to build a large company starting from nothing, which I think he still owns 50% of (the other 50% being owned by his initial angel investor).

So, 5 out of these 6 millionaires got there almost completely by luck. Ironically, at least 2 of these 5 are absolutely convinced that anyone in the US who is smart and works hard will become a millionaire.

Comment author: SforSingularity 29 August 2009 07:32:07PM 2 points [-]

So, 5 out of these 6 millionaires got there almost completely by luck

that 5 of your friends all got very lucky seems very unlikely to me. Perhaps you have a biased sample of intelligent, motivated friends (i.e. more intelligent and motivated than average).

Comment author: saturn 30 August 2009 10:35:45AM *  1 point [-]

that 5 of your friends all got very lucky seems very unlikely to me.

Perhaps Phil prefers making friends with people who are already millionaires.

Comment author: PhilGoetz 31 August 2009 11:18:09PM *  -1 points [-]

It was not just luck in the stock-options cases. They wouldn't have gotten lucky if they weren't capable, motivated people. So, biased sample, yes. But, still, lucky.

saturn - No, I didn't initially know that they were millionaires.

Comment author: SforSingularity 31 August 2009 11:22:40PM 1 point [-]

So, biased sample, yes. But, still, lucky.

how do you know? Do you have 5 other capable, motivated friends who shot for millionairedom and failed over and over again?

Comment author: pjeby 29 August 2009 10:22:20PM 1 point [-]

Ironically, at least 2 of these 5 are absolutely convinced that anyone in the US who is smart and works hard will become a millionaire.

Which two?

Comment author: wedrifid 28 August 2009 07:30:09PM -1 points [-]

They can. A lazy person recognizing they are lazy and becoming a driven go-getter type would be a far less plausible outcome.

Lazy + intelligent + flexible ethics is a reasonable combination for becoming a millionaire (or a prison inmate).

Comment author: Vladimir_Nesov 28 August 2009 06:48:50PM 0 points [-]

I can make a stronger complaint than that: that the more we find out about ourselves, the emptier we feel.

How generally does your statement apply? How do you know it? There are wrong ways to learn, but also good ones. Also, the level of happiness usually can't be systematically affected.

Comment author: SforSingularity 28 August 2009 07:19:22PM 2 points [-]

Also, the level of happiness usually can't be systematically affected.

It is probably true that learning things is unlikely to make you any happier or unhappier than you were to start with.

Comment author: SforSingularity 28 August 2009 07:18:03PM 0 points [-]

How generally does your statement apply?

It seems to be fairly general. Can you point to a set of news items about human nature that makes us feel better about ourselves? I can point to lots that probe human self-delusion, weakness and mechanicity.