Also, the difference in crime rate might amount to something like "if you walk through the areas once a day, you'll be mugged on average once every ten years or once every thirty years."
There may be a difference between the rate at which a resident (who's probably at similar income to other residents in the area, and perceived as an insider) would be mugged, and the rate at which a visitor (seen as likely to be carrying more money, and perceived as a outsider, and the residents won't have to come face to face with them or their families again) would be mugged.
That said, as I noted in a previous comment, there are definitely cases where people are prone to designate a place as "dangerous" when it's not actually statistically more...
There's a lot of background mess in our mental pictures of the world. We try and be accurate on important issues, but a whole lot of the less important stuff we pick up from the media, the movies, and random impressions. And once these impressions are in our mental pictures, they just don't go away - until we find a fact that causes us to say "huh", and reassess.
Here are three facts that have caused that "huh" in me, recently, and completely rearranged minor parts of my mental map. I'm sharing them here, because that experience is a valuable one.