Of course.

I agree that

"Interesting point. I'm not entirely clear how you arrived at that position. I'd like to look up some detail questions on that. Could you provide references I might look at?"

doesn't make clear that the other holds another position and that the reply may just address the validity of the evidence.

But even then shouldn't you see it at least as weak evidence and thus believe X at least a bit more strongly?