There may be a difference between the rate at which a resident (who's probably at similar income to other residents in the area, and perceived as an insider) would be mugged, and the rate at which a visitor (seen as likely to be carrying more money, and perceived as a outsider, and the residents won't have to come face to face with them or their families again) would be mugged.

That said, as I noted in a previous comment, there are definitely cases where people are prone to designate a place as "dangerous" when it's not actually statistically more dangerous than where they already live. The fact that visiting these neighborhoods may be more dangerous per time spent than living in them doesn't make it likely that people who refuse to visit them are assessing risk realistically.