In what sort of evidence space can physical brain computation yielding complexity limits count as bits of evidence factoring into expected physical outcomes (such as the exponential smallness of the spectral gap of NP-hard-Hamiltonians from the quantum adiabatic theorem)?

These sorts of questions are really tough. However, for most useful purposes it seems that we don't need to ask this (difficult) question. The main issue is that our empirical investigation of our universe suggests that the physical laws don't allow us to do lots of efficient computation (most obviously, whatever the physical laws are, it seems that we can't easily use them to compute NP complete problems in polynomial time, and certainly can't do so for #P). So most of the conclusions that Scott is arriving at come in some sense from empirical observations about the world around us. They just aren't a class of observations that we're used to thinking about as observations of the physical world (they are much more abstract than something like "all electrons have the same rest mass").