You are correct that it's not a solved problem, but any new theory of face recognition, or in general any proposed computational model of the visual cortex, can be trivially tested: have it look through pictures and recognize faces, and compare it to how well untrained humans can recognize faces.

Isn't this rather like building a few cars, comparing how well they run, and calling it physics?