I think you're pessimistic about tech regression.
Assuming survival of some libraries, I think basically any medium-sized functional village (thousands of people, or hundreds with a dash of trade) is adequate to maintain iron age technology. That's valuable enough that any group that survived in a fixed location for more than a couple years could see the value in the investment. (You might not even need the libraries if the right sort of person survived; I suspect I could get a lot of it without that, but it would be a lot less efficient.)
It doesn't take all that much more beyond that to get to some mix of 17th to 19th century tech. Building a useful early 19th-century machine shop is the work of one or two people, full time, for several years. Even in the presence of scavenging, I think such technology is useful enough that it won't take that long to be worth spending time on.
Basically I think anything that's survivable is unlikely to regress to before 17th century tech for a period longer than a few years.
If only one village survive, it will use scavenging of nearby city for decades and during this time it will lost most high educated people.
But lets assume that scavenging economy is no possible, may be this village survive on remote island, which is probable, in case of pandemic.
In this case the main problem will be economy of scale, chemistry and raw materials. Even 19 century economy is based on available coal, iron ore, copper and some other things. Most of them require very specific people, knowledges and instruments to find and produce.
And also they ...
Levels of global catastrophes: from mild to extinction
It is important to make a bridge between existential risks and other possible risks. If we say that existential risks are infinitely more important than other risks, we put them out of scope of policymakers (as they can’t work with infinities). We could reach them if we show x-risks as extreme cases of smaller risks. It could be done for most risks (with AI and accelerator's catastrophes are notable exceptions).
Smaller catastrophes play complex role in estimating probability of x-risks. A chain of smaller catastrophes may result in extinction, but one small catastrophe could postpone bigger risks (but it is not good solution). The following table presents different levels of global catastrophes depending of their size. Numbers are mostly arbitrary and are more like placeholders for future updates.
http://immortality-roadmap.com/degradlev.pdf