Well I don't think it makes sense to regret one's entire past and be satisfied with merely that. You want to draw specific lessons from your past. An ideal agent might not need regret of course, being able to learn from past mistakes without a feeling of regret toward a specific event which gave rise to the general lesson. But I think humans might find it useful to have an event serve as a reminder of a lesson learned.

We can interpret Caplan's "no regret" (perhaps too charitably) as "my past does not contain any lessons wrt. me behaving in a certain way in order to have my children be a certain way". But this leaves room for other lesson-specific regret wrt. other genuine lessons.

As for the massive counterfactual of "Caplan having behaved even minisculely different in his past", I think it's quite useless and hence meaningless, at least with respect to Caplan and his children. It doesn't help him better raise his children, for example. It's like how not every English sentence corresponds to a meaningful statement.