I second this. Friends selection usually involves having some mutual interests: math, music, movies, parties, whatever. The focus of the activity you do together will mostly invole those interests, therefore you will put more effort into getting "better" at them, if only because you're spending more time practicing.

I don't think the opposite can be true: of you hate physics, you can't just hang out with a phisicist to get better at it. I regularly hang out with my oldest group of firends, and none of them has ever expressed interest in knowing more about my thesis on the Higgs Boson - or the law of conservation of momentum for what matters. On the other hand, I can have more challenging conversation with my colleagues on this topic, but, while many of them are also excellent musicians, I never thought about getting to practice with my guitar again.