Most of your resource base collapses because one person is going to have to work MUCH harder to provide enough food for themselves -- fields go unplanted, vegetables don't get tended, wild game is much more dangerous to hunt by oneself, and one cannot expect any assistance with gathering. Even a small number of people used to an agriculture-enriched lifestyle are going to be hit much harder.

Both Clark and Tainter (Collapse of Complex Civilizations) disagree with this claim as stated. A massive reduction in the population means that the survivors get increased per-capitas because the survivors move way back along the diminishing marginal returns curve and now have more low-hanging fruit (sometimes literally). In fact, Tainter argues that complexity often collapses because the collapse is the only way to increase per-capita wealth. Hunter-gatherers spend much less time per calorie than do advanced agriculturalists eg.

The surprise here is that while there is wild variation across forager and shifting cultivation societies, many of them had food production systems which yielded much larger numbers of calories per hour of labor than English agriculture in 1800, at a time when labor productivity in English agriculture was probably the highest in Europe. In 1800 the total value of output per man-hour in English agriculture was 6.6 pence, which would buy 3,600 kilocalories of flour but only 1,800 kilocalories of fats and 1,300 kilocalories of meat. Assuming English farm output was then half grains, onequarter fats, and one-quarter meat, this implies an output of 2,600 calories per worker-hour on average.32 Since the average person ate 2,300 kilocalories per day (table 3.6), each farm worker fed eleven people, so labor productivity was very high in England. Table 3.13 shows in comparison the energy yields of foraging and shifting cultivation societies per worker-hour. The range in labor productivities is huge, but the minimum average labor productivity, that for the Ache in Paraguay, is 1,985 kilocalories per hour, not much below England in 1800. The median yield per labor hour, 6,042 kilocalories, is more than double English labor productivity.

Or

...ranging from a modest 1,452 kilocalories per person per day for the Yanomamo of Brazil to a kingly 3,827 kilocalories per person per day for the Ache of Paraguay. Some of this is undoubtedly the result of errors in measuring food consumption. But the median is 2,340, implying that hunter-gatherers and subsistence agriculturalists ate as many calories as the median person in England or Belgium circa 1800. Primitive man ate well compared with one of the richest societies in the world in 1800. Indeed British farm laborers by 1863 had just reached the median consumption of these forager and subsistence societies.

(Quotes brought to you by my Evernote; it's a pain in the ass to excerpt all the important bits from a book, but it certainly pays off later if you want to cite it for various assertions.)