The bit in Josephus was a complete forgery, likely inserted by Eusebius (well-documented as a chronic and unapologetic liar for the Church).
Neither Pliny nor Tacitus wrote anything about Jesus - they wrote about Christians, the existence of whom is not in question. Further, it's well documented that Tacitus was tampered with.
The notability is that Jesus was claimed to be known to all, with scribes following him about.
Look, these objections really are standard, long-standing and pretty well documented. Reading up in the area is absolutely fascinating. Wikipedia is a half-decent start.
To break up the awkward silence at the start of a recent Overcoming Bias meetup, I asked everyone present to tell their rationalist origin story - a key event or fact that played a role in their first beginning to aspire to rationality. This worked surprisingly well (and I would recommend it for future meetups).
I think I've already told enough of my own origin story on Overcoming Bias: how I was digging in my parents' yard as a kid and found a tarnished silver amulet inscribed with Bayes's Theorem, and how I wore it to bed that night and dreamed of a woman in white, holding an ancient leather-bound book called Judgment Under Uncertainty: Heuristics and Biases (eds. D. Kahneman, P. Slovic, and A. Tversky, 1982)... but there's no need to go into that again.
So, seriously... how did you originally go down that road?
Added: For some odd reason, many of the commenters here seem to have had a single experience in common - namely, at some point, encountering Overcoming Bias... But I'm especially interested in what it takes to get the transition started - crossing the first divide. This would be very valuable knowledge if it can be generalized. If that did happen at OB, please try to specify what was the crucial "Aha!" insight (down to the specific post if possible).