Monotremes are mammals, yet do not share the property of giving live birth.

"Hair... may be greatly reduced in the Cetacea (i.e. dolphins), where it is found as a few scattered bristles about the lips or often present only in the young." W. J. Hamilton, American Mammals.

Heterocephalus glaber, the naked mole rat, is a mammal despite being poikilothermic.

As DanArmak notes, mammals are explicitly not the set of all species fulfilling a particular set of (external) criteria. They are defined by descent.

To the extent that this is useful, great. My point was merely that there is no external fact of the matter that requires drawing the boundaries where they currently are.

So when “ordinary people…get it wrong” by believing that dolphins are fish, it’s a little hard to blame them.