"Whether you have herpes" is not as clearly-defined a category as it sounds. The blood test will tell you which types of HSV antibodies you have. If you're asymptomatic, it won't tell you the site of the infection, if you're communicable, or if you will ever experience an outbreak.

I had an HSV test a while ago (all clear, thankfully), and my impression from speaking to the medical staff was that given the prevalence and relative harmlessness of the disease, (compared to, say, HIV or hepatitis or something), the doubt surrounding a positive test result was enough of a psychological hazard for them to actively dissuade some people from taking it, and many sexual health clinics don't even offer it for this reason.