Yes, that's the problem - my appetite is initially suppressed and then I wind up incredibly hungry.

That makes sense. And given how degree of hunger influences what kind of food we desire you'll end up craving a bunch of carbohydrates later rather than whatever healthy food you had planned at the meal times.