You could always exercise at home. Pushups, squats, and crunches are all easy muscle building exercises there. I personally love "DDR" (video game) for this - a used copy of the game, a cheap dance mat, and a used PS/2 can be had for under $100 pretty easily.

I also occasionally just jog or walk especially fast during my normal day. I've never had anyone comment on it, even when I do this downtown. Amongst other things, I'm moving faster than them ;)

I would second the suggestion of "self modify so you don't mind if people think you're high", but I don't think that's the simplest way to get exercise :)