When discussions about such topics are opened on LW, often the logical next step would be to ask about the more general underlying problems that give rise to these situations, instead of just focusing on the arguments about particular problems in isolation. (And even without a concrete motivation, such questions should directly follow from LW's mission statement.) Yet I see few, if any attempts to ask such general questions on LW, and my occasional attempts to open discussion along these lines, even when highly upvoted, don't elicit much in terms of interesting arguments and insight.
Your own points have struck me as on the mark; but I haven't had much to add.
There are some interesting general comments that I could make based on my experience in the mathematical community in particular. I guess here I have some tendency toward self-preservation myself; I don't want to offend acquaintances who might be cast in negative life by my analysis. (Would be happy to share my views privately if you're interested though.) I guess my attitude here is that there's little upside to making my remarks public. The behaviors that I perceive to be dysfunctional are sufficiently deeply entrenched so that whatever I would say would have little expected value.
The main upside would be helping others attain intellectual enlightenment, but although I myself greatly enjoy the satisfaction of being intellectual enlightenment, I'm not sure that intellectual enlightenment is very valuable from a global perspective. Being right is of little use without being influential. In general the percentage of people who are right (or interested in being right) on a given topic where a contrarian position is right is sufficiently small so that the critical mass that it would take to change things isn't there and nor would an incremental change in this percentage make a difference.
The reason why the above point has so much weight in my mind is that despite my very high interest in learning about a variety of things and in forming accurate views on a variety of subjects; I haven't achieved very much. It's not clear whether having accurate views of the world has been more helpful or harmful to me in achieving my goals. The jury is still very much out and things may change; but the very fact that it's possible for me to have this attitude is a strong indication that knowledge and accurate views on a variety of things can be useless on their own.
The best cure against such prideful attitudes is to ask yourself what you have to show in terms of practical accomplishments and status if you're so much more rational and intellectually advanced than ordinary people. If they are so stupid and delusional to be deserving of such intolerance and contempt, then an enlightened and intellectually superior person should be able to run circles around them and easily come out on top, no?
Regarding:
As an illustration, we can take an innocent and mainstream problematic topic like e.g. the health questions of lifestyle such as nutrition, exercise, etc. These topics have been discussed on LW many times, and it seems evident that the mainstream academic literature is a complete mess, with potential gems of useful insight buried under mountains of nonsense work, and authoritative statements of expert opinion given without proper justification. Yet I see no attempt to ask a straightforward follow-up question: since these areas operate under the official bureaucratic system that's supposed to be guaranteed to produce valid science, then what exactly went wrong? And what implications does it have for other areas where we take the official output of this same bureaucratic system as ironclad evidence?
I made a comment that you may find relevant here; I would characterize nutrition/exercise/etc. as fields that are obviously important and which therefore attract many researchers/corporations/hobbyists/etc. having the effect of driving high quality of researchers out of the field on account of bad associations.
Another factor may be absence of low hanging fruit (which you reference in your top level post); it could be that the diversity of humans is sufficiently great so that it's difficult to make general statements about what's healthy/unhealthy.
I agree with what you said about main stream fields being diluted, but offer an interesting corollary to that. Economic motives compel various gurus and nutritionists to make claims to the average joe, and the average joe, or even the educated joe cannot sort through them. However, if one looks in more narrow fields, one can obtain more specific answers without so much trash. For example, powerlifting. This is not a huge market nor one you can benefit financially from that much. If one is trying to sell something or get something published, he can't just ...
When I found Less Wrong and started reading, when I made my first post, when I went to my first meetup….
It was a little like coming home.
And mostly it wasn’t. Mostly I felt a lot more out of place than I have in, say, church youth groups. It was hard to pinpoint the difference, but as far as I can tell, it comes down to this: a significant proportion of the LW posters are contrarians in some sense. And I’m a conformist, even if I would prefer not to be, even if that’s a part of my personality that I’m working hard to change. I’m much more comfortable as a follower than as a leader. I like pre-existing tradition, the reassuring structure of it. I like situations that allow me to be helpful and generous and hardworking, so that I can feel like a good person. Emotionally, I don’t like disagreeing with others, and the last thing I have to work hard to do is tolerate others' tolerance.
And, as evidenced by the fact that I attend church youth groups, I don’t have the strong allergy that many of the community seem to have against religion. This is possibly because I have easily triggered mystical experiences when, for example, I sing in a group, especially when we are singing traditional ‘sacred’ music. In a previous century, I would probably have been an extremely happy nun.
Someone once expressed surprise that I was able to become a rationalist in spite of this neurological quirk. I’ve asked myself this a few times. My answer is that I don’t think I deserve the credit. If anything, I ended up on the circuitous path towards reading LessWrong because I love science, and I love science because, as a child, reading about something as beautiful as general relativity gave me the same kind of euphoric experience as singing about Jesus does now. My inability to actual believe in any religion comes from a time before I was making my own decisions about that kind of thing.
I was raised by atheist parents, not anti-theist so much as indifferent. We attended a Unitarian Universalist church for a while, which meant I was learning about Jesus and Buddha and Native American spirituality all mixed together, all the memes watered down to the point that they lost their power. I was fourteen when I really encountered Christianity, still in the mild form of the Anglican Church of Canada. I was eighteen when I first encountered the ‘Jesus myth’ in its full, meme-honed-to-maximum-virulence form, and the story arc captivated me for a full six months. I still cry during every Good Friday service. But I must have missed some critical threshold, because I can’t actually believe in that story. I’m not even sure what it would mean to believe in a story. What does that feel like?
I was raised by scientists. My father did his PhD in physical chemistry, my mother in plant biology. I grew up reading SF and pop science, and occasionally my mother or my father’s old textbooks. I remember my mother’s awe at the beautiful electron-microscope images in my high school textbooks, and how she sat patiently while I fumblingly talked about quantum mechanics, having read the entire tiny physics section of our high school library. My parents responded to my interest in science with pride and enthusiasm, and to my interest in religion with indulgent condescension. That was my structure, my tradition. And yes, that has everything to do with why I call myself an atheist. I wouldn’t have had the willpower to disagree with my parents in the long run.
Ultimately, I have an awfully long way to go if I want to be rational, as opposed to being someone who’s just interested in reading about math and science. Way too much of my motivation for ‘having true beliefs’ breaks down to ‘maybe then they’ll like me.’ This is one of the annoying things about my personality, just as annoying as my sensitivity to religious memes and my inability to say no to anyone. Luckily, my personality also comes with the ability to get along with just about anyone, and in a forum of mature adults, no one is going to make fun of me because I’m wearing tie-dye overalls. No one here has yet made fun of me for my interest in religion, even though I expect most people disagree with it.
And there’s one last conclusion I can draw, albeit from a sample size of one. Not everyone can be a contrarian rationalist. Not everyone can rebel against their parents’ religion. Not everyone can disagree with their friends and family and not feel guilty. But everyone can be rational if they are raised that way.