given how you describe yourself as kind of straddling the line there, I suppose whether you describe yourself as "libertarian" or not might depend on how, when, and by whom you're asked.
How would a journalist from say the New York Times describe your political views if he full had access to them?
An article from the Wall Street Journal. The original title might be slightly mind-killing for some people, but I found it moderately interesting especially considering that many LessWrongers formed part of the data set for the study the article talks about and a large fraction of us identified as libertarian on the last survey.
Inside the Cold, Calculating Libertarian Mind
The original paper.
Understanding Libertarian Morality: The Psychological Roots of an Individualist Ideology