Okay, I think I should take a minute to clarify where exactly we disagree. Starting from your conclusion:

So our current understanding of the scientific method must be incomplete: there is some way of obtaining reliable theories about the world other than the standard theorize/predict/test loop.

This by itself isn't objectionable: of course you can move your probability distribution on your future observations closer to reality's true probability distribution without controlled experiments. And Bayesian inference is how you do it.

But you also say:

Now observe that human children can learn to make very reliable predictions. So they must be doing some sort of science. But they don't make controlled experiments

I agree that children learn how to solve AI-complete problems, including reliable prediction in this environment (and also face-recognition, character-recognition, bipedal traversal of a path barring obstacles, etc.). But you seem to have already concluded (too hastily, in my opinion) that the answer lies in a really good epistemology that children have that allows them to extract near-maximal knowledge from the data in their experiences.

I claim that this ignores other significant sources of the knowledge children have, which can explain how they gain (accurate) knowledge even when it's not entailed by their sense data. For example, if some other process feeds them knowledge -- itself gained through a reliable epistemology -- then they can have beliefs that reflect reality, even though they didn't personally perform the (Bayes-approximating) inference on the original data.

So that answers the question of how the person got the accurate belief without performing lots of controlled experiments, and the problem regresses to that of how the other process gained that knowledge and transmitted it to the person. And I say (based on my reading of Pinker's How the Mind Works) that the most likely possibility for the "other process" is that of evolution.

As for the transmission mechanism, it's most likely the interplay between the genes, the womb, and reliably present features of the environment. All of these can be exploited by evolution, in very roundabout ways, to increase fitness. For example, the DNA/womb system can interact in just the right way to give the brain a certain structure, favorable to some "rituals of cognition" but not others.

This is why I don't expect you to find a superior epistemology by looking at how children handle their experiences -- you'll be stuck wondering why they make one inference from the data rather than another that's just as well-grounded but wrong. Though I'm still interested in hearing why you think you've made progress and what insights your method has given you.