I've known a thin guy who would do this to seem bulkier. More common is the thermal long-sleeved undershirt with a t-shirt over it, which is also better on the warmth front.

Ensure that you're good with color coordination, then just do it. It's more likely to be the good kind of unconventional than the weird kind.