Associating skyscrapers with New York doesn't constitute approval for either.

You might truly abhor racial prejudice and still recognize that black people are usually portrayed in a negative light, and thus "blackness" and "bad" are associated in your mind. That's not being prejudiced as such.