The problem isn't fantasizing about winning the lottery. The problem is paying to fantasize about winning the lottery. The expected value of the ticket is negative, but you can fantasize for free.

EDIT: But even then, there are more valuable uses of your time than fantasizing about something that will never happen.

This means you don't lose anything if you choose to fantasize about anything you feel like.

You do lose something; the opportunity cost of spending time fantasizing. If you get enough out of fantasizing to offset that cost, then by all means do it, but for most people there are other valuable things they could do that are just as fun, like fantasizing as a means of fleshing out book ideas.