The argument for improper priors is that the resulting posterior distributions work well in various senses. No one uses improper priors for prediction -- the resulting prior predictive densities are improper too, so it's impossible.

Here's the argument by which I justify improper priors to myself when I use them: in cases where I have very little prior information but highly informative data, the proper prior will be essentially proportional to the improper prior in the region of high likelihood. Then using the improper prior as an approximation results in an approximate posterior which gives results that differ only negligibly from the results I would have obtained with the "correct" proper prior.