Actually, good thing you asked, because I gave wrong information in my original comment. Chisholm is an expert on the problem of the criterion, but I was actually thinking of William Alston in my comment. Here's two papers, one by Alston and one by another author that I've referenced in the past and found useful:

William P. Alston. Epistemic Circularity. Philosophy and Phenomenological Research, 47(1):1, sep 1986.

Jonathan Dancy. Ethical Particularism and Morally Relevant Properties. Mind, XCII(368):530– 547, 1983.

From your post:

This is circular, but is this necessarily a problem? If your choice is a circular justification or eventually hitting a level with no justification, then the circular justification suddenly starts looking pretty attractive.

I think the coherentist article I linked to has some useful perspective here. The quote below is from a section in that article on regress. The first paragraph outlines a view similar to yours and raises an important objection against the circular justification view. The 2nd paragraph raises a potential response.

What is the coherentist’s response to the regress? The coherentist can be understood as proposing that nothing prevents the regress from proceeding in a circle. Thus, A can be a reason for B which is a reason for C which is a reason for A. If this is acceptable, then what we have is a chain of reasons that is never-ending but which does not involve an infinite number of beliefs. It is never-ending in the sense that for each belief in the chain there is a reason for that belief also in the chain. Yet there is an immediate problem with this response due to the fact that justificatory circles are usually thought to be vicious ones. If someone claims C and is asked why she believes it, she may reply that her reason is B. If asked why she believes B, she may assert A. But if prompted to justify her belief in A, she is not allowed to refer back to CC which in the present justificatory context is still in doubt. If she did justify A in terms of C nonetheless, her move would lack any justificatory force whatsoever.

The coherentist may respond by denying that she ever intended to suggest that circular reasoning is a legitimate dialectical strategy. What she objects to is rather the assumption that justification should at all proceed in a linear fashion whereby reasons are given for reasons, and so on. This assumption of linearity presupposes that what is, in a primary sense, justified are individual beliefs. This, says the coherentist, is simply wrong: it is not individual beliefs that are primarily justified, but entire belief systems. Particular beliefs can also be justified but only in a secondary or derived sense, if they form part of a justified belief system. This is a coherence approach because what makes a belief system justified, on this view, is precisely its coherence. A belief system is justified if it is coherent to a sufficiently high degree.

Of course, the key coherentist claim is that an entire belief system can be justified without individual beliefs being justified because the property of being justified is a property of belief systems that emerges from the coherence of multiple beliefs.

Sure you can carry on trying but you are not guaranteed to succeed. You could go increasingly meta without finding a loop.

If meta-justifications are just not adhoc you could employ them against loops. If I was unsatisafied with a level of justification being circular I could insist that there must be a further level of intuitions that warrant the situation which themselfs don't have the loopy nature.

I don't really think that infinite chains are a good approach but I am not convinced that the investigation is cast in solid enough of logic that it makes explicit the reasons to take its finding seriously. A method of exhaustion with open vents is comparatively weak.