I think you have an extra negation in your last sentence.

Also, I can't quite see why a direct link between mental models and their referents is any stronger evidence of living in a simulation than anything else is. If it turns out that that's how reality works, then that's how reality works, and it's just as real in that case as the direct link between mass and the strength of gravitational attraction.

I sort of have a similar intuition to yours, but it seems clear to me that in my case I'm simply reacting to how weird the psychic-powers thing sounds. (Where weird -> unreal in my head.) But of course reality is full of things that are weird until they are commonplace.