I eventually identified that the most important secondary criterion for rationality here, which I think is what makes physical materialism different from mainstream rationality, is that belief should only be supported by the positive existence of (physical) evidence.
So, it's true that all the evidence I use is physical, but I don't even know what it would mean for evidence to be non-physical. My brain is physical, evidence that can interact with my brain is physical, mental events that my brain experiences are physical. If I had some sort of soul that was not made of quarks and gluons or anything else that physicists talk about, but it interacted with my physical brain, by that interaction, I would consider that the soul had to be physical, even if it is a different sort of physical stuff that anything else we know about.
For example, a theist has lots of 'evidence' for their faith, including affirming emotional and mental states, the example of the religious conviction of their friends and family, and the authority of religious leaders. However, physical materialism would train us to reject all that as evidence of a personal God
So, we would not actually dismiss the emotional and mental states as non-physical, because they, having been observed by a physical system, clearly are physical.
there are other explanations for these observations.
That points to part of the correct explanation. The emotional and mental states are not considered strong evidence because, although they are likely given that God exists, they are not unlikely given that God does not exist.
Yes, certainly. I hope you don't think I disagree with any of your points.
Well, actually, if I was required to take issue with any of them, it would be with the importance of probability in deciding that the existence of God is not compelling. I don't think probability has much to do with it, especially in that perhaps in a counterfactual reality there ought to be a high probability that he exists. But what is compelling is that once you are detached from the a priori belief he is present, you notice that he isn't. For me, it isn't so much a question of &...
I've spent so much time in the cogsci literature that I know the LW approach to rationality is basically the mainstream cogsci approach to rationality (plus some extra stuff about, e.g., language), but... do other people not know this? Do people one step removed from LessWrong — say, in the 'atheist' and 'skeptic' communities — not know this? If this is causing credibility problems in our broader community, it'd be relatively easy to show people that Less Wrong is not, in fact, a "fringe" approach to rationality.
For example, here's Oaksford & Chater in the second chapter to the (excellent) new Oxford Handbook of Thinking and Reasoning, the one on normative systems of rationality:
Is it meaningful to attempt to develop a general theory of rationality at all? We might tentatively suggest that it is a prima facie sign of irrationality to believe in alien abduction, or to will a sports team to win in order to increase their chance of victory. But these views or actions might be entirely rational, given suitably nonstandard background beliefs about other alien activity and the general efficacy of psychic powers. Irrationality may, though, be ascribed if there is a clash between a particular belief or behavior and such background assumptions. Thus, a thorough-going physicalist may, perhaps, be accused of irrationality if she simultaneously believes in psychic powers. A theory of rationality cannot, therefore, be viewed as clarifying either what people should believe or how people should act—but it can determine whether beliefs and behaviors are compatible. Similarly, a theory of rational choice cannot determine whether it is rational to smoke or to exercise daily; but it might clarify whether a particular choice is compatible with other beliefs and choices.
From this viewpoint, normative theories can be viewed as clarifying conditions of consistency… Logic can be viewed as studying the notion of consistency over beliefs. Probability… studies consistency over degrees of belief. Rational choice theory studies the consistency of beliefs and values with choices.
They go on to clarify that by probability they mean Bayesian probability theory, and by rational choice theory they mean Bayesian decision theory. You'll get the same account in the textbooks on the cogsci of rationality, e.g. Thinking and Deciding or Rational Choice in an Uncertain World.