It's certainly not either/or (nor did you necessarily suggest it was).
This doesn't include non-economic reasons. People who think more or less people should be married, should or shouldn't consume certain substances or sell or see particular things. The idea that there is one true way or, at least, some things that can't be a part of anyone's way has a long history.
Libertarianism is hard because most people want to be in other people's business.
I'll provide a tentative answer that isn't supported by any scientific research I'm aware off, only my own observations as an autodidact History student, so take it with several grains of salt. With that disclaimer in place, here's a hypothesis I've been nurturing for a few years already.
My insight takes as its main premise the fact traditional societies have organized in a hierarchical pattern that, in its broad strokes, varies little between places and times. This structure usually has four main classes, sometimes less, sometimes more, but when the number varies its usually either because two of the four are considered as one, or because sub-types of one are distinguished as their own individual classes. Additionally, the means of access (or lack thereof) to those classes, aka class mobility, vary a lot, ranging from self-attribution, to rigid by-birth caste rules, and passing through lots of other options, including meritocratic-like assignment via standardized tests, as was and remains the practice in China. Myself, I look at these classes as a broad way of classifying people in light of their psychologically-motivated core goals.
The four classes are:
a) Those who seek power, with a strong will to rule others, and set the rules others must follow.
Someone with this psychological profile this doesn't care for wealth, although wealth may be come as a side-effect of their exercise of power.
They feel fine if they're poor but are obeyed.
Abstract and pure knowledge doesn't appeal to them, if they need to know something they ask a summary from those who know.
b) Those who seek wealth, with a strong will to take calculated risks if the payoff is worth it.
Someone with this psychological profile doesn't care for power, although power may be, for them, a means to wealth, and so they may go for it.
They feel fine if they're as wealthy as possible, and even more so if their risk taking pays off in the form of even more wealth.
Their take about knowledge is similar to that of "a".
c) Those who seek knowledge and/or wisdom, with a strong will to learn as much as possible.
Someone with this psychological profile doesn't care for either power or wealth and prefer to avoid dealing with those at all if possible, although in the real world it's unavoidable, so they face the need to deal with both the powerful and the wealth lest their pursuit of knowledge is hindered.
They feel fine if they're learning, and respected for their knowledge. This may come in the form of dogmatic knowledge (think a religious theologians or jurist) or in an open minded, contrarian way (the kind of intellectual the first would hate, and vice-versa).
When dealing with "a" or "b", they either work for them, or are persecuted by them, but rarely want to exercise either kind of power. If they do so, it's by necessity, as a means to secure their access to learning. Usually their power, if any, is indirect, by means of setting the stage within which both "a" and "b" think.
d) Those who seek stability and safety, with a strong will to resist anything that might imperil these.
Someone with this psychological profile doesn't care for holding power, and neither for the risk taking involved in pursuing great wealth. They seek a stable life, within the boundaries of socially accepted mores, and the knowledge they pursue is for the practical uses of their profession, or a hobby.
They feel fine when they're respected by their community, for their work, and feel the finest when things work exactly as they've always worked. Changes, particularly quick, broad changes, stress them out. Therefore, they're usually conservative, but not in an ideological way, in a practical way.
When dealing with "a" or "b", they tend to obey as long as they don't feel their safety and stability threatened. When dealing with "c", they tend to agree if what they're hearing fits with their worldview, or to disagree in varying degrees of incisiveness when what they're hearing challenges their worldview. They're accepting of change if and only if it's brought slowly, and only if there's a clear benefit for their safety and stability, or at the very least no downside for them and theirs.
These are the four classes/castes typically found in traditional societies all over the world. At least, those I read about do fit, with minor variations.
So, let's take this, assume these classes are invariant, meaning that no matter how much society changes there will always be people whose deep psychological profiles are such that they follow these four end-goals, and contrast that with Libertarianism.
i) For type-a individuals, Libertarianism is something they clearly don't want. A minarchist society is one in which their power is severely reduced, while what they seek is to maximize it.
This fits with your Elite Support Hypothesis.
ii) For type-b individuals, Libertarianism is an "it depends" proposition. They don't care much about the principles, but they do care about the consequences:
If Libertarianism is implemented in such a way that doesn't prevent unrestricted wealth growth, including monopolization efforts, then they're for it in that specific context.
If Libertarianism challenges their wealth-seeking and the payoff of their risk-taking, they're against it.
And even if they at first are for it, but later conditions change and suddenly Socialism becomes the way for them to grow their wealth, that's what they'll go for.
So, this both fits and doesn't fit with your Elite Support Hypothesis.
iii) For type-c individuals, being for or against Libertarianism is a matter of intellectual preference. Some are drawn to it, some aren't, some are disgusted by it. It depends on how it fits or doesn't fit their other intellectual interests and preferences, specially if Economics isn't their main field of study.
As a rule of thumb, I venture type-cs tend to be anti-Libertarian in practice because in a minarchist society there are way less possibilities of dedication to pure intellectual pursuits compared to scenarios in which the State funds them without requiring immediate practical applications to arise from that funding. Sure, some type-cs have interests so well aligned with those of type-bs in their business capacity that they will always easily find corporate R&D funding, but for the majority of type-cs it's either going for type-as good will (public-funded research, government grants); for type-bs prestige support (tenured positions in private universities, partisan-NGO/think tank positions etc.); for type-ds direct support (acting sage/priest, journalism, infotainment host etc.); or giving up and going for a soul-crushing type-b or type-d lifestyle.
So, this both fits and doesn't fit, but mostly fit, with both your Elite and Non-Elite Support Hypothesis, as type-cs may be one, the other, both, or none, depending on context.
iv) Finally, for type-d individuals, Libertarianism is a downright scary thing, as Libertarianism embraces constant change on a massive scale as a very positive value, which goes against type-ds seeing constant, big changes as a safety hazard full of the very risks that must be avoided.
This fits your Non-Elite Support Hypothesis.
In summary, I'd say Libertarianism is more likely to be preferred by type-bs, and opposed by types a, c and d.
As for other economic systems, they have different distributions. Taking into account the Economics schools I've studied at least a little about, this would result in this table, with "+", "-" and "0" meaning, respectively, "in favor of/advantageous to", "opposed/disadvantageous to", and "neutral/indifferent towards", and exclamations meaning "see below":
(!) I group in this things like European-style Social Democracy, Northern-European countries' brand of Capitalistic Socialism, US-style Liberalism (known elsewhere as Social-Liberalism), German-style Ordoliberalism etc. Or, to put it another way, big State, big corporations, huge taxes, big welfare state.
(!!) "+" for the majority of small players, "-" for the big ones.
(!!!) "+" if it's already established, "-" if it isn't and it'd require major societal changes, "0" otherwise.
(!!!!) Disclaimer: my own preference.
I hope this helps, and I'm all ears (eyes) for criticisms.
EDIT: Replaced exclamation marks for asterisks, as Markdown was turning those into italics and bolds.
I think both of those hypotheses contain some truth, and I also think it's a mistake to limit either of them to modern times. In pre-modern times the governing institutions may not have looked like a state in the modern sense, but life involved as much as or much more coercion controlling individual behavior and imposing unchosen obligations on us. Sometimes this was a matter of mutual dependence for survival, sometimes of enforcing norms for peaceful coexistence without constant infighting, and so on.
If there is a fundamental problem, I think it's the one Aristotle pointed out:
Man is by nature a social animal; an individual who is unsocial naturally and not accidentally is either beneath our notice or more than human. Society is something that precedes the individual. Anyone who either cannot lead the common life or is so self-sufficient as not to need to, and therefore does not partake of society, is either a beast or a god.
Libertarianism takes as a foundational principle that society cannot justly compel or coerce us to accept obligations we do not choose. This generally includes a large degree of distrust in any institution labeled "government." But to function in society without governing institutions, we would need to put much more trust in individuals to manage their own affairs without harming others (we have no non-coercive dispute resolution mechanisms efficient and fair enough to use them consistently and frequently in normal life), to efficiently negotiate and enforce agreements among one another, to control their own behavior and make good decisions (or else stand by and watch them fail and suffer without the benefit of scaled-up systems to help people recover from failures). This would be much easier if people were on average much smarter and saner, but still hard, since essentially you're removing a lot of the defaults we rely on in our own and others' behavior and forcing deliberate choice.
I'm fairly sure there could be a society organized around libertarian principles that would be great to live in, but I'm not sure there's a practical stepwise path for getting from here to there. Robin Hanson has done a pretty good job posing hypothetical institutions that could do many of the things governments currently do, without much government involvement, but still more than many libertarians claim to want.
Also, even if we had such a society, it's hard to make it stable over time. Some people are going to have more success than others, the kinds of success that lead to wealth and the power wealth can buy, including power over people who choose to work for you or otherwise form economic relationships with you. This includes inherited wealth you get by accident of birth. I'm consistently amazed by the degree to which the libertarians I know IRL are so opposed to government coercing anyone in any way, while also having no problem with the power private actors like corporations have to coerce their customers and employees. And in fact most people don't want to live in a world where they're free to act as they choose but powerless to access any actual choices they find even remotely compatible with what they consider a good life.
Tyler Cowen earlier this year wrote about the intellectual demise of libertarianism (Cowen 2020). I'm here wondering: aside from its normative status, is libertarianism unsustainable for reasons intrinsic to itself? Said differently, is it true that there are mechanisms intrinsic to libertarianism that forestall its ability to exist?
To stimulate discussion, the following two hypotheses came to mind that support this notion. Both of these hypothesis posit that actors within the polity will vote for politicians that increase state capacity and thus prevent/terminate libertarianism.
Elite Support Hypothesis. Economic growth in modern societies demands certain investments and solutions to collective action problems that only a state can provide. Wealthy elites will support politicians that increase state capacity to provide these services and these politicians will gain power.
Non-Elite Support Hypothesis. Economic inequality will rise to a point where non-elites will support politicians favoring a welfare state and these politicians will gain power.