This is true about 90-95% of the time. I have on a few occasions voted for "the other party", usually based on specific policy differences they have with their party, but occasionally because the candidate in my party is too odious, and the admittedly poor evidence I have the opponent's character is favorable.

I often I vote third party though, especially when the election doesn't seem close.