People control how companies operate - even though they don't understand all the components involved (in particular the brains).

Any idea that you have to understand all the components of a system in order to exert a high level of control over it thus appears to have dubious foundations.

There are alternative approaches to producing predictable systems which basically involve lots of unit tests.

Testing is in vogue in the software engineering world. Few systems are simple enough to prove much about their behaviour. So: to make sure they behave as they are intended, they are intensively tested. It seems likely that machine intelligence will be no different.