Yes, in some cases, there is no (after the fact) non-controversial set of issues to use to determine how effective an expert is. Which means that I can't convince the general public of how much they should trust the expert, but I can still figure out how much I should trust em by looking at their positions that I can evaluate.

There is also the possibility of saying something about such an expert based on correlations with experts whose predictions can be non-controversially evaluated.