Right, and there's the issue of whose fault the difficulty is. Sure, the student might not really be trying. But also, the teacher may not be explaining in a way that speaks to the learner's natural fluency. A method that works for the geeky types won't work work for more neurotypical types.

For my part, I never have trouble explaining high school math to those who haven't completed it, even if they're told that trig, calculus, etc. is hard. It's because I first focus on finding out where exactly their knowledge deficit is and why the subject matter is useful. Of course, teachers don't have the luxury of one-on-one instruction, but yes, how you present the material matters greatly.