I think it was more a perception of a general lack of competence on the part of Perry during the debate then either one of those things.

People aren't just voting on the issue, at least not for president; trying to find someone who will intelligently and competently run a large bureaucracy and make decisions about new issues and foreign affairs on the fly is probably just as important. If someone seems like they're not competent enough to do the job properly, then that seems like a rational reason to vote against them.

(Now, if Perry is actually incompetent or if he just came across that way is an issue that could be debated, but I think it's a rational thing for voters to consider.)