I've been thinking about this problem a bit. I think that every futurist paper should include a section where it lists, clearly, exactly what counts as a failure for this prediction. In fact, that would be the most important piece of the paper to read, and those with the most stringent (and short term) criteria for failure should be rewarded.

And, in every new paper, the author should list past failure, along with a brief sketch of why the errors of the past no longer apply here. This is for the authors themselves as much as for the readers - they need to improve and calibrate their predictions. Maybe we could insist that new papers on a certain subject are not allowed unless past errors in that subject are addressed?

Of course, to make this all honest and ensure that errors aren't concealed or minimized, we should ensure that people are never punished for past errors, only for a failure to improve.

Now, if only we could extend such a system to journalists as well... :-)