My impression is that you're atypical in this. I watched Star Trek, read books with teleporting, and so on, and while I was completely unbothered by its portrayal in fiction (mostly because, I think, they deliberately showed it as non-destructive), I nonetheless have a visceral aversion to the prospect of personally being teleported by destructive means. (The issue is also, for example, explicitly discussed in Dan Simmons' Ilium and most of the characters are suitably horrified when they discover that the teleporters they've been using are destructive, although they get over it in the interests of expediency.)