I don't have a good resource for you - I've had too much math education to pin down exactly where I picked up this kind of logic. I'd recommend set theory in general for getting an understanding of how math works and how to talk and read precisely in mathematics.

For your specific question about the mean, it's the only number such that the sum of all (samples - mean) equals zero. Go ahead and play with the algebra to show it to yourself. What it means is that if you go off of the mean, you're going to be more positive of the numbers in the sample than you are negative, or more negative than positive.