Perhaps this is a known issue, but since I haven't seen it discussed, I thought I'd mention that images don't seem to work in some (all?) of the old posts imported from Overcoming Bias. See for example:

Timeless Physics

The first few pics in that particular post is available from an external server if you click them, but I don't see them inline. The last picture seems to have been hosted at Overcoming Bias, and is no longer accessible.

Moral skepticism is not particularly impressive as it's the simplest hypothesis. Certainly, it seems extremely hard to square moral realism with our immensely successful scientific picture of a material universe.

The problem is that we still must choose how to act. Without a morality, all we can say is that we prefer to act in some arbitrary way, much as we might arbitrarily prefer one food to another. And...that's it. We can make no criticism whatsoever about the actions of others, not even that they should act rationally. We cannot say that striving for truth is any better than killing babies (or following a religion?) anymore than we can say green is a better color than red.

At best we can make empirical statements of the form "A person should act in such-and-such manner in order to achieve some outcome".

Some people are prepared to bite this bullet. Yet most who say they do continue to behave as if they believed their actions were more than arbitrary preferences.

My position may be one of those you criticize. I believe something that bears an approximation to "morality" is both worth adhering to and important.

I think a particular kind of morality helps human societies win.

Morality, as I understand it, consists of a set of constraints on acceptable utility functions combined with observable signals of those constraints.

Do I believe that this type of morality is in any sense ultimately correct? No. In a technical sense, I am a complete and total moral skeptic.

However, I do think publicly-observable moral behavior is useful for coordination and cooperation, among other things. To the extent that this makes us better off--to the extent it makes me better off--I would certainly think that even a moral skeptic might find it interesting.

Perhaps LWers are "too uncritical toward their moral prejudices." But it's at least worth examining which of those "moral prejudices" are useful, where this doesn't conflict with other, more deeply held values.

Finally, morality broadly enough construed is a condition of rationality: if morality is taken to simply be your set of values and preferences, then it is literally necessary to a well-defined utility function, which is itself (arguably) a necessary component of rationality.

I'm continually surprised that so many people here take various ideas about morality seriously. For me, rationality is very closely associated with moral skepticism, and this view seems to be shared by almost all the rationalist type people I meet IRL here in northern Europe. Perhaps it has something to do with secularization having come further in Europe than in the US?

The rise of rationality in history has undermined not only religion, but at the same time and for the same reasons, all forms of morality. As I see it, one of the main challenges for people interested in rationality is to explore how to live without morality. Many "rationalists" instead go into denial and try to construct some supposedly rational form of morality, more often than not suspiciously similar to the traditional ideas. I'm not sure whether or not Eliezer's metaethical project is an example of this, but in any case he is commendable for taking the issues very seriously. Most other LW:ers seem to be far too uncritical toward their moral prejudices.

This is really from times before OB, and might be all too obvious, but the most important thing I’ve learned from your writings (so far) is bayesian probability. I had come in touch with the concept previously, but I didn’t understand it fully or understand why it was very important until I read your early explanatory essays on the topic. When you write your book, I’m sure that you will not neglect to include really good explanations of these things, suitable for people who have never heard of them before, but since no one else has mentioned it in this thread so far, I thought I might.

Expecting Short Inferential Distances

One of many posts that gave me a distinct concept for something I previously had been only vaguely aware of, and this one kept coming back to me all the time. By now, I don’t think it’s an extreme exaggeration to say that I make use of this insight every time I communicate with someone, and of all the insights I picked up from OB, this might be the one I most frequently try to explain to others. It doesn’t seem like the most important thing, but for some reason, it immediately struck me as the most frequently useful one.

I can’t remember a time when I was not very much concerned with rationality. I think my father (a neuroscientist) encouraged those kinds of ideas from the time I was learning to speak my first few words, always reasoning with me, nudging me to think straight. I developed a deep interest in science from about the age of five and there was never any competition from other ways of viewing the world. Things like game theory and heuristics and biases came to me much later (when studying economics), and although I was excited about it, it didn’t really rock my world. I had always been searching for tools with which to improve my own thinking, these just happened to be unusually powerful ones.

Although I don’t remember any awakening to rationalism, I do remember some early clashes with irrationalism, which I think was quite formative. From the beginning, I had taken rationalism for granted. As I started to interface with the world outside my family, I realized that the norm was in fact massive irrationalism, and this drove me crazy. The prime example was when I encountered religion.

My parents were second-generation atheists, and socialized exclusively with other atheists. Also, I had the good fortune to grow up in a country where a vast majority of the population is nonreligious. For these reasons I didn’t even know that such a thing as religion existed until I was about eight years old. At that point, I joined a classmate from elementary school to an after school activity group arranged by a church. I came home afterwards and told my parents about the stories I’d heard about this person called Jesus. They said simply that if I wanted to go there and listen, that was okay, but it was important that I realize from the beginning that the stories were just stories, as in any book of fiction. That was the first and last thing my parents ever tried to teach me about religion.

Some time later I realized that religious people actually believed the biblical stories, without any kind of reasonable evidence, and I found this absolutely horrifying. I remember feeling deeply offended that such ignorance could exist, and still it took a long while for the full scale of the offense to sink in. A couple of years later I slept over at a neighboring family’s apartment and was shocked when, shortly before going to sleep, the parents of the family expected me to get down on my knees and pray. They were equally shocked when I said that I had no idea how to pray. In any case, I was thoroughly disgusted by this first encounter with actual people actually practicing religion, and ever since, I’ve had to struggle just to keep a straight face when I meet religious people. (My parents advised me to take it easy and tolerate religious people, saying that some of them actually were good and decent people, it was just that they happened to have this slightly “childish” aspect to their characters, which should be tolerated in the same way as, for example, low intelligence.)

I never read Overcoming Bias for the rationality stuff. Although I have certainly learned a lot from these posts, I have never felt that they were very revolutionary for me personally (allthough I guess they would be for most of the world). My main interest here is Mr Yudkowsky himself. I have a life-long interest in the nature of genius, and reading the things he writes seems to me an unusually unobstructed view into the mind of a living and ever-developing genius. What he happens to be writing about at the moment (I’ve been following his work for over five years now) is of secondary importance.