Kaczynski is the only psychopath I've really read much about, so maybe I really did extrapolate his seeming rationality onto other psychopaths

I don't know too much about him other than the basics ("he argued that his bombings were extreme but necessary to attract attention to the erosion of human freedom necessitated by modern technologies requiring large-scale organization").

I think that his concerns are valid, but I don't see how the bombings help him achieve the goal of bumping humanity off that path. Perhaps he knew he'd get caught and his manifesto would get attention, but a) there's still a better way to achieve his goals, and b) he should have realized that people have a strong bias against serial killers.

The reason I think his concerns are valid is because capitalism tries to optimize for wanting, which is sometimes quite different from liking. And anecdotally, this seems to be a big problem.

That would have to be some kind of bias; out of curiosity how would you label it? Maybe survivorship bias? Or availability heuristic? Anchoring? Or maybe even all of the above?

I'm not sure what the bias is called :/. I know it exists and there's a formal name though. I know because I remember someone calling me out on it LWSH :)

Nope, there's really not, but another thing I've realized from reading SSC is that a major component of great writing (and teaching) is the sharing of relevant, interesting, relatable examples to help an idea.

Yes, I very much agree. At times I think the articles on LW fail to do this. Humans need to have their System 1's massaged in order to understand things intuitively.

Anyway, Scott allows people to go easy on themselves for contributing less to the world than they might like, relative to their innate ability. Can we also go easy on ourselves relative to innate conscientiousness?

Idk. This seems to be a question involving terminal goals. Ie. if you're asking whether our innate conscientiousness makes us "good" or "bad".

When I think of morality this is the/one question I think of: "What are the rules we'd ask people to follow in order to promote the happiest society possible?". I'm sure you could nitpick at that, but it should be sufficient for this conversation. Example: the law against killing is good because if we didn't have it, society would be worse off. Similarly, there are norms of certain preference ratios that lead to society being better off.

I don't think we'd be better off if the norm was to have, say equal preference ratios for everyone in the world. Doing so is very unnatural would be very difficult, if not impossible. You have to weigh the costs of going against our impulses against the benefits that marginal conscientiousness would bring.

I'm not sure where the "equilibrium" points are. Honestly, I think I'd be lying to myself if I said that a preference ratio of 1,000,000,000:1 for you over another human would be overall beneficial to society. I suspect that subsequent generations will realize this and look at us in a similar way we look at Nazis (maybe not that bad, but still pretty bad). Morality seems to "evolve" from generation to generation.

Personally, my preference ratios are pretty bad. Not as bad as the average person because I'm less scope insensitive, but still bad. Ex. I eat out once in a while. You might say "oh well that's reasonable". But I could eat brown rice and frozen vegetables for very cheap and be like 70% as satisfied, and pay for x meals for people that are quite literally starving.

But I continue to eat out once in a while, and honestly, I don't feel (that) bad about it. Because I accept that my preference ratios are where they are (pretty much), and I think it makes sense for me to pursue the goal of achieving my preferences. To be less precise and more blunt, "I accept that I'm selfish".

And so to answer your question:

Can we also go easy on ourselves relative to innate conscientiousness?

I think that the answer is yes. Main reason: because it's unreasonable to expect that you change your ratios much.

Yeah, this is sooo real. On a logical level, it's easy to recognize my scope insensitivity. On a "feeling" level, I still don't feel like I have to go out and do something about it.

It's great that you understand it on a logical level. No one has made much progress on the feeling level. As long as you're aware of the bias and make an effort to massage your "feeling level" towards being more accurate, you should be fine.

But I don't want to admit my preference ratios are that far out of whack; I don't want to be that selfish.

Why?

I think that answering that exploring and answering that question will be helpful.

Try thinking about it in two ways:

1) A rational analysis of what you genuinely think makes sense. Note that rational does not mean completely logically.

2) An emotional analysis of what you feel, why you feel it, and in the event that your feelings aren't accurate, how can you nudge them to be more accurate.

This is BY FAR the best guidance anyone has ever given me in my life.

Wow! Thanks for letting me know. I'm really happy to help. I've been really impressed with your ability to pursue things, even when it's uncomfortable. It's a really important ability and most people don't have it.

I think that not having that ability is often a bottleneck that prevents progress. Ex. an average person with that ability can probably make much more progress than a high IQ person without it (in some ways). It's nice to have a conversation that actually progresses along nicely.

Anyway, another question for you. You know how you said we care only about our own happiness? Have you read the part of the sequences/rationality book where Eliezer brings up someone being willing to die for someone else? If so, what did you make of it? If not, I'll go back and find exactly where it was.

I think I have. I remember it being one of the few instances where it seemed to me that Eliezer was misguided. Although:

1) I remember going through it quickly and not giving it nearly as much thought as I would like. I'm content enough with my current understanding, and busy enough with other stuff that I chose to put it off until later. Although I do notice confusion - I very well may just be procrastinating.

2) I have tremendous respect for Eliezer. And so I definitely take note of his conclusions. The following thoughts are a bit dark and I hesitate to mention them... but:

a) Consider the possibility that he does actually agree with me, but he thinks that what he wrote will have a more positive impact on humanity (by influencing readers)

b) In the case that he really does believe what he writes, consider that it may not be best to convince him otherwise. Ie. he seems to be a very influential person in the field of FAI, and it's very much in humanities interest for that person to be unselfish.

I haven't thought this through enough to make these points public, so please take note of that. Also, if you wouldn't mind summarizing/linking to where and why he disagrees with me, I'd very much appreciate it.

Edit: Relevant excerpt from HPMOR

They both laughed, then Harry turned serious again. "The Sorting Hat did seem to think I was going to end up as a Dark Lord unless I went to Hufflepuff," Harry said. "But I don't want to be one."

"Mr. Potter..." said Professor Quirrell. "Don't take this the wrong way. I promise you will not be graded on the answer. I only want to know your own, honest reply. Why not?"

Harry had that helpless feeling again. Thou shalt not become a Dark Lord was such an obvious theorem in his moral system that it was hard to describe the actual proof steps. "Um, people would get hurt?"

"Surely you've wanted to hurt people," said Professor Quirrell. "You wanted to hurt those bullies today. Being a Dark Lord means that people you want to hurt get hurt."

Sorry, I feel like I'm linking to too many things which probably feels overwhelming. Don't feel like you have to read anything. Just thought I'd give you the option.