Almost everyone finds an explicit refusal to answer offputting. Don't do it. But that doesn't mean that you should actually answer. Usually a good choice is to answer a different question, such as to make them an offer.