Definitely explore what infinite willpower means, since it would certainly be possible to hurt myself since it presumably includes a lot of ability to ignore pain.
Would I give up Minesweeper or find out what it's like to play Mindsweeper while completely ignoring all internal and external distractions? Unfortunately, the latter is more attractive.
One good thing about infinite willpower is that I'd be better able to track the effects of various choices. For example, I'd have infinite ability to give up recreation, but that might mean my mind is less usefully flexible in a week or a month.
Keeping your muscles flexed all the time is not a good idea. I recommend reading Ralston's Zen Body-Being (not a great title, but a lot of good stuff in the book) about the importance of putting the least effort into martial arts.
I would definitely explore putting less into low intensity pleasures.
Back to experiment-- this is really important, because, since infinite willpower doesn't exist, it wouldn't be obvious, for example, how much socializing supports other goals.
If infinite willpower seems to be leading to good results after five or ten years, I'd start researching how to make it available to other people.
If (as a result of infinite willpower or by other means), life is made generally good-- prosperity and health for all, what would preference utilitarians do with infinite willpower?
For the past few years my willpower has been steadily increasing. If it lasts and I use it to accomplish something noteworthy, I might write a post about it.
Anyway, what should a rational preference utilitarian with infinite willpower do? Assume that there are no negative effects (unhappiness, stress) with using this willpower, and that they can control their emotions at will.
Clearly they should work a lot more, not spend time on recreation (movies, TV, games), stand instead of sitting, etc..
What else? Should they listen to music? Should they keep their muscles flexed 24/7 ? What should they learn, where would they have the most relative advantage? How much time would be worth spending on social interaction?
I can figure out these things on my own, but those questions are important and good ideas are very valuable.