If you value your belief that's there are no ghost then it's irrational to be scared by ghosts.
Are you talking about "real" ghosts? You shouldn't be afraid of real ghosts because they don't exist, not because you value your belief that there are no ghosts. Why should beliefs have any value for you beyond their accuracy?
If there's such a small chance of your vote making a difference in the election, you should be comfortable voting for trump.