One distinction is that they most smart people who have looked at the SIAI have come away not thinking they are crazy

There is a strong selection effect. Most people won't even look too closely, or comment on their observations. I'm not sure in what sense we can expect what you wrote to be correct.