Well, this is something that can be tested experimentally. It could be statistically tested whether the results of the top predictors resemble random noise.

Some people spend incredible amounts of time on internet, reading about stuff that interests them. I can imagine they could make good predictions in their area. (And there should be a "no vote" option for questions outside of their area.)

Wikipedia exists, despite it doesn't pay its contributors, unlike other encyclopedias. And there is some good stuff there. Also bad stuff... but that's what the competition between predictors could fix.

There is probably a limit on how difficult things can be predicted. But it could be higher than we imagine. Especially if the predictions become popular, so for many topics there would be predictors whose hobby it is.

There are some technical details to solve, e.g. whether the predictor's prestige will be global, or topic-dependent. (To prevent people from systematically giving 10 great prediction in topic X, and then 1 bad but very visible in topic Y.) But that's like having multiple StackExchange accounts.