Why do you think GRE is in any way corrolated with innate IQ? That couldn't be further from the truth. GRE scores are corrolated with how much and how well you studied for the GRE, and that's it. Did you use a prep programme? I bet you the 32% above you did.

Regarding coursework, it sounds like you finally reached the level where you're starting to get challenged. Congrats. The question is how you respond to that challenge. Sorry, but in that sense I think "grow up" is the right answer.

When do I give up? In these sorts of cases, when I'm no longer interested in the outcome. You still want to do physics and math, right?