It's just that physicalists take the controlled and repeated findings of physics and neuroscience as being stronger evidence than their own subjective experience is.

This is not a straightforward matter. There are people who deny the existence of colors, of time, of any sort of will, or any sort of subjective experience, on the basis of "physicalism" or "science". Physics and neuroscience actually contain no such thing as "subjective experience". Do you therefore conclude that there is no such thing at all? No, you believe it's there and it has a semi-obscure relationship to the physical facts, and you use what you hear from science to adjust your beliefs about the nature of that relationship, and also to adjust your attitude towards the naive beliefs that arise naturally from your experience.

What's actually going on in the minds of people who make private attitude adjustments in order to be good little physicalists, is itself best described as a subjective process. The complex of beliefs peculiar to this site, as recently summarized in the Gospel According to Zed, are a great case study in such self-shaping of subjectivity. It's subjectively experienced, it's subjectively willed, and it's done in response to a subjectively imagined idea of what Science is saying. This is true both for the more spectacular ontological commitments that people make and break, and for the epistemological adjustments performed in response to the psychological demonstration of fallibility of judgment. And I'd say it usually has about as much basis as the belief of someone who thinks there is a gay gene because they remember hearing about it on TV. Science can be wrong, the popular description of what science says can be wrong, and the private implications that a person draws from this can be wrong.