Well, "stupid" may be a little unfair in the case of Order members or Lucius, but there's clearly a big intelligence gab between Harry and Riddle on the one hand and everyone else on the other hand. It's not just lack of positive evidence that tells us this. We have Dumbledore's own testimony that he (and the entire rest of the Order of the Phoenix) was totally outmatched against Voldemort in the first wizarding war.