I've actually never heard of diffusion for planning. Do you have a reference?

A diffusion model for text generation (like Diffusion-LM) still has the training objective to produce text from the training distribution, optimizing over only the current episode - in this case a short text.