If you are commuting downtown during rush hour, being with other human beings is a downside - it's quite oppressive, actually. And you probably won't get a seat, which means napping is out, and reading is more of a hassle.

I'm in somewhat of an ideal situation, commute-wise - I work just outside the city and live inside, so I commute in the opposite direction of traffic. But I've had to commute downtown occasionally and it's way more exhausting.