It depends on what you call "delusional". Just to be clear: I'm not arguing that justification is impossible, but that "at bottom" all our beliefs rest on uncertain axioms that are provisionally treated as certain, but which aren't justified. Aesthetic considerations such as boring vs. interesting, elegant vs. obtuse, natural vs. tortured then loom much larger in the actual part they play in determining beliefs than supposed certainty about axioms.

Additionally there's a problem with whether it's actually possible to truly believe something without justification. If your beliefs don't make contact with your experiences then what exactly is it you're believing in?