There is indeed a difference between "we have observed good evidence of X" and "there is something out there that, had we observed it, would be good evidence of X".

Even so, absence of observed evidence is evidence of absence.

How strong it is depends, of course, on how likely it is that there would be observed evidence if the thing were real. (I don't see anyone ignoring that fact here.)