Conversely, we know the LHC is not going to destroy the world, because nature has been banging particles together at much higher energy levels for billions of years. If that sufficed to destroy the world, it would already have happened, and any people you might happen to meet from time to time would be figments of a deranged imagination.

I don't know if this argument is quite valid. AFAIK there are specific differences in the way the particles are being banged together in the LHC. One of those being that at point of impact their relative speed to the earth will be reduced to 0 so any black hole created will be stationary in relation to the earth. Also there might actually be some places in the universe where black holes have been generated in similar fashion but we don't know about it.

The truth is I don't know enough physics to really understand the potential dangers and I assume that neither do most people who write about the LHC. All I can do is hope that the people who are in charge know what they are doing.