Sounds like a sort of double-counting. "If I stopped believing the bad side of town is the bad side, then I might go over there and get mugged!"
I don't think that's necessarily it, I think it's more like "avoiding that section of town costs me so little that I don't want to bother to think about whether avoiding it makes sense".
There's a lot of background mess in our mental pictures of the world. We try and be accurate on important issues, but a whole lot of the less important stuff we pick up from the media, the movies, and random impressions. And once these impressions are in our mental pictures, they just don't go away - until we find a fact that causes us to say "huh", and reassess.
Here are three facts that have caused that "huh" in me, recently, and completely rearranged minor parts of my mental map. I'm sharing them here, because that experience is a valuable one.