Yes, certainly. I hope you don't think I disagree with any of your points.
Well, actually, if I was required to take issue with any of them, it would be with the importance of probability in deciding that the existence of God is not compelling. I don't think probability has much to do with it, especially in that perhaps in a counterfactual reality there ought to be a high probability that he exists. But what is compelling is that once you are detached from the a priori belief he is present, you notice that he isn't. For me, it isn't so much a question of "existence" but failed promise.
So, it's true that all the evidence I use is physical, but I don't even know what it would mean for evidence to be non-physical. My brain is physical, evidence that can interact with my brain is physical, mental events that my brain experiences are physical. If I had some sort of soul [...] I would consider that the soul had to be physical, even if it is a different sort of physical stuff that anything else we know about.
It seems you might have forgotten, if you were ever familiar, with the pre-materialist understanding of the concept of 'non-physical'. Personally, I've forgotten. It's hard to hang on to a concept that is rendered inconsistent. I don't think it was as coarse as 'these thoughts make me happy, so they must be true' or that feelings and 'mental states' are considered to be independent somehow of scientific analysis. Though maybe. Maybe it was the idea that a person could figure stuff out about the world by thinking in a certain way about what "ought" to be, where 'ought' is pulled from some Platonian ideal value system.
Yes ... that you can sit in a chair and decide that circles exist, and would exist even if there didn't happen to be any. That's there another source of knowing.
I've spent so much time in the cogsci literature that I know the LW approach to rationality is basically the mainstream cogsci approach to rationality (plus some extra stuff about, e.g., language), but... do other people not know this? Do people one step removed from LessWrong — say, in the 'atheist' and 'skeptic' communities — not know this? If this is causing credibility problems in our broader community, it'd be relatively easy to show people that Less Wrong is not, in fact, a "fringe" approach to rationality.
For example, here's Oaksford & Chater in the second chapter to the (excellent) new Oxford Handbook of Thinking and Reasoning, the one on normative systems of rationality:
Is it meaningful to attempt to develop a general theory of rationality at all? We might tentatively suggest that it is a prima facie sign of irrationality to believe in alien abduction, or to will a sports team to win in order to increase their chance of victory. But these views or actions might be entirely rational, given suitably nonstandard background beliefs about other alien activity and the general efficacy of psychic powers. Irrationality may, though, be ascribed if there is a clash between a particular belief or behavior and such background assumptions. Thus, a thorough-going physicalist may, perhaps, be accused of irrationality if she simultaneously believes in psychic powers. A theory of rationality cannot, therefore, be viewed as clarifying either what people should believe or how people should act—but it can determine whether beliefs and behaviors are compatible. Similarly, a theory of rational choice cannot determine whether it is rational to smoke or to exercise daily; but it might clarify whether a particular choice is compatible with other beliefs and choices.
From this viewpoint, normative theories can be viewed as clarifying conditions of consistency… Logic can be viewed as studying the notion of consistency over beliefs. Probability… studies consistency over degrees of belief. Rational choice theory studies the consistency of beliefs and values with choices.
They go on to clarify that by probability they mean Bayesian probability theory, and by rational choice theory they mean Bayesian decision theory. You'll get the same account in the textbooks on the cogsci of rationality, e.g. Thinking and Deciding or Rational Choice in an Uncertain World.