Since a house is inevitably both a place to live and an investment, it would seem to be appropriate to treat it as both. (Unless doing so spoils your enjoyment of it as a place to live, or something. For what it's worth, I've always thought of houses both ways, have never noticed such a negative effect, and have always been happy with the results on both counts. But I've been pretty fortunate.)

[EDITED to add: I agree that the extra mortgage interest you'll pay is a genuine extra cost -- and that, not the $35k price difference or whatever, is what you should be weighing against whatever you're paying the extra for. How the two figures relate to one another depends a lot on the mortgage interest rate, how quickly you repay, etc.]