It definitely is a mileage varies thing, though getting rid of or weakening the horny layer (the layer of dead skin cells that protects the living skin beneath) can have non-obvious consequences. For example, you may experience more sun damage than you otherwise would and thus accumulate wrinkles more quickly.

Going one day without showering isn't enough to see improvement, but if you consistently take a sans douche and avoid soap, hard scrubbing, and hot water, you might see long-term improvements.

On the other hand, some people like hot showers a lot. It may not be worth it to you.