If you refer to "fish" rather than a particular species (or at least to "red fish" vs "white fish"?) then I have to wonder which varieties you've tried. There are significant enough differences between tuna, cod, salmon, and tilapia, for instance, that I would not be surprised to find a person whose liked/disliked any combination of the four.

Do all kinds of fish have the same health effects, BTW? What about molluscs and crustaceans?