I should think that wandless magic, while possible, is at least far less powerful than wanded; presumably with that advantage, as well as anti-Disapparition jinxes, and perhaps other pre-cast defensive spells, a competent Auror should be able to hold and defeat any wizard. It might well be possible, for instance, to have an ongoing magic-triggered overpowered Somnium enchantment cast on the prison, that ignores the wands of specifically keyed-in wizards. Or something.