In some cases that's true - one of the things willpower is used for is dealing with situations where you don't want to make the effort of doing the thing, but you do want something that you'll get as a result of having done the thing. It also applies in situations where you do want to do the thing, but you're fatigued in some way, or distracted, and it probably applies in other kinds of situations that aren't coming to mind at the moment.

ETA: There does seem to be a significant difference, at least for me, between what it feels like to use willpower to do something I truly don't want to do (as in, it's not worth the reward) vs. other uses of willpower. I'm not sure how best to describe it, though. A first approximation is: For things that I truly don't want to do, there's no situation where I can do them without using willpower - even thinking of the reward doesn't help. (That's how I figured out I needed to leave my old job at the nursing home - I went a month or so without ever actually wanting to be there, with no sign of it getting better.)