Why is that probability a priori miniscule? Something like 50% of people in the US believe in some form of psi and there's an entire academic discipline devoted to researching it, many of the scientists in which are reputable and work at prestigious universities (and many have a background in physics). No psi effects that I know of are theoretically ruled out given our uncertainty about QM (even the retrocausal stuff fits in with many interpretations, e.g. the transactional interpretation) and even QM is incomplete. There is a history of unseen forces turning out to have huge harness-able effects, like nuclear power or electricity. There is also a general perception that the more science we do the less agency we see in the world, which is some evidence against psi. Overall I think the prior should be low but not miniscule, and that since the value of information is so high because psi is a phenomenon which if it exists would falsify many ingrained inductive biases, it would be very wise to look into the only phenomenon that could at least in principle falsify those biases.
(I might be rationalizing my conclusion somewhat here, as I do in fact know that psi probably exists. Before carefully looking into the issue I had basically the same beliefs as you do. Note that I find the probable existence of psi really annoying and didn't start looking into the issue because I wanted to believe. Practical epistemology would be much easier in a psi-less world,)
I assign very low probability to psi for several reasons. The first is evolution. If psi is an evolved feature of human brains, then it should be universal in humans and provide fitness advantages, but I've seen no evidence of these fitness advantages.
The second is that it is convenient. Humans like fantasizing about having superpowers. That is by far a simpler explanation for widespread belief and academic study.
The third is the economic argument, as explained by Randall Munroe here.
The fourth you've already mentioned; psi is an almost textbook example of...
[Post redacted 'cuz I unfairly and carelessly misrepresented someone's views (Eliezer's). The messages of this post was: disbelief that aliens visit Earth in spaceships is a bad reason not to look into ufology. My apologies for this ugly post.]