The phenomenon that you allude to (great researchers setting a high standard for themselves and feeling inadequate relative to it even while doing extremely good work by most people's standards) is a real one, but in the above passage Grothendieck is in part (explicitly) comparing himself with other people who he knows.

A version of your comment that takes this into account is that he may have been comparing his speed with that of the greatest mathematicians in the world (e.g. his close correspondent Jean-Pierre Serre, who I believe to be the youngest Fields medalist in history).