Following the sequence link at the top, I found this similar post, which has an impressive list of references. You include there this paper by Timothy Williamson. It seems to me an oversight you don't mention the paper's argument at all, as it's a sustained critique of the position you're representing.

The basic idea is that the kind of doubts about intuitions you raise are relevantly similar to more familiar forms of philosophical scepticism (scepticism about the external world, etc). I understand Williamson sees a dilemma: either they are mistaken for the same reasons familiar scepticism is mistaken (Williamson's position, to which most of the paper is dedicated), or the doubts undermine way more than its proponents think they do.

It'd be great to hear your summary of the argument there, and what you consider to be its flaw(s).

If you like Williamson, check out also this excellent bit on naturalism.