The assumption that we can better determine toxicity with our current understanding of human biology than thousands of years of natural selection seems questionable, but peanuts are certainly a good lower bound on selection's ability.

I also don't have much confidence that the parties responsible for safety testing are particularly reliable, but that's a loose belief.