First, a "crazier and more subcultural" look is essentially a form of peacocking. Like all peacocking, it can be extremely effective, but it's difficult to pull off competently and tends to backfire badly if done in a less than stellar way.

I am indeed talking about peacocking. I agree with you about the risks of peacocking, and perhaps I didn't acknowledge them clearly enough.

So I definitely wouldn't recommend it to a typical guy, who almost certainly has much more advisable options than attempting peacocking.

Actually, I think peacocking of some sort is one of the options I would most advise to a typical guy attempting to improve social and romantic success. It can quickly help him get more attention and new sorts of reactions (e.g. compliments that increase his confidence).

Even attempting a more conventional elaborate hairstyle can backfire. You know when a man tries dressing sharply but instead of looking sharp ends up projecting that bad "I'm trying too hard" look? (I don't have in mind being badly dressed by any clear standard, but rather giving off that vague impression that it's not his natural image, and it just doesn't fit him in some hard-to-describe way.) My impression is that it's even easier to fail in a similar way by attempting a sophisticated haircut.

I know that sort of look, and I've certainly been guilty of it myself. Yet my view is that trying and failing can often look better than not trying, especially for a guy who is below average to start with. My hypothesis is that a lot guys with average or below average style could drop on a crazy hairstyle and get better reactions, even if they looked like they were trying too hard, and their hair made no sense with the rest of their clothing. Guys who are already at least above average in style, or guys in older or more conservative/conformist cultures might not benefit so much.

Good peacocking is powerful. But even messy trial-and-error while learning peacocking may actually be better than what many guys are already doing.

Otherwise, however, I have the impression that men's efforts put in hairstyle rapidly hit diminishing returns,

Effort on hairstyle hits diminishing returns once you separate yourself from the pack of average guys you are competing with. How much effort that takes depends on (sub)culture.

except perhaps in a peacocking context.

What other sort of context is there?

Certainly it seems to me that countless other aspects of looks and behavior are far more important outside of these basic limits.

There are many aspects of looks and behavior that are more important than peacocking, but in my experience, peacocking has a pretty good cost-benefit ratio.

Peacocking synergizes with other form of social self-improvement, even when done badly. Even though peacocking has a learning curve, it may actually be easier than certain other social self-improvement skills for certain types of guys.

Compare fashion to other skills that a guy might be learning. For shy, introverted males, they have a lot of things to focus on in the "field," such as posture, body language, voice, content of speech, etc... which can easily be overwhelming. Everything has a learning curve. The advantage of fashion (including hair) is that you can cache it before stepping into the field, which is a massive boon: fire and forget. Furthermore, in contrast to conversation, fashion is broadband and travels at the speed of light.

Psychologically, what traits are fashion skills loaded on?

• pattern recognition
• spatial skills and artistic ability
• creativity
• technical knowledge of tools and procedures

Fashion is the medium by which these abilities can be converted into social status. Psychologically, fashion is actually right up the alley of creative and visual introverted systemizers. Culturally, fashion isn't seen that way because it's associated with women and gay men.