Would you know what to look for?

For the most part

Also, can you write an AI bot that would have a decent success rate against humans, by finding and exploiting the standard wetware bugs?

Not for any interesting opponent. I can't even write a 'real' chatbot. The only reason I get the results I do is because I immediately force them into a binary yes/no response and then make sure they keep saying yes .