The black box is made of humans and might be tested the usual way when (human-designed) WBE tech is developed.

Ok, I think I misunderstood you earlier, and thought that your idea was similar to Paul Christiano's, where the FAI would essentially develop the WBE tech instead of us. I had also suggested waiting for WBE tech before building FAI (although due to a somewhat different motivation), and in response someone (maybe Carl Shulman?) argued that brain-inspired AGI or low-fidelity brain emulations would likely be developed before high-fidelity brain emulations, which means the FAI would probably come too late if it waited for WBE. This seems fairly convincing to me.