Is that a personal impression or a memory of research that you can't find at the moment?

Memory of research. I don't know enough trust-fund kids to have a personal impression.

"Adversity builds character"

We're talking about a slightly different thing: the need for external motivation. The point isn't that for proper character development you should spend your youth going to school barefoot in the snow uphill both ways. The point rather lies in establishing the pattern and the expectation that if you want something, you should be prepared to pay for it in work (time, effort, resources), that some desires are too expensive this way, and that the amount of work you're prepared to do limits the things you can get.

If, as a kid or a teenager, you don't get any practice in limiting your wants and in pushing yourself to get something, you do not develop the appropriate "muscles" and so grow up lacking in abilities to control your desires and push yourself over the internal whining of "but I don't wanna".

Note that I'm not arguing that poverty is better. I am arguing that growing up without any financial constraints is not all rainbows and unicorns and brings with it its own set of problems.