Although there are no long-term scientific studies of calorie restriction in humans, there are religious groups, cults, and ascetics who voluntarily practice calorie restriction or intermittent fasting. Presumably there have been tens or hundreds of thousands of people who have practiced calorie restriction throughout most of their adult lives. There were/are probably also groups that involuntarily practice calorie restriction - servants, slaves, prisoners, or people who simply regularly don't have enough to eat.

If calorie restriction has a dramatic effect on life expectancy in humans, shouldn't we expect to observe extended life expectancy in at least some groups? Or would each of these groups likely have some mitigating circumstances that would shorten their lifespans, such as lack of medicine?

With an hour on Google, I found some references to Okinawa, to monks on Mount Athos, and to similar groups. In no case was there a reasonable claim of life expectancy over 90 (which would represent just a 10% improvement over life expectancy in Japan).

This paper reviews the evidence on calorie restriction in humans and other animals, including discussion of religious fasting, but there's no evidence there of fasting extending lifespan.

I found a few other sources where people asked this question (or made this point as an attack on CR), but I haven't yet found any good answers on the subject, and didn't find any discussion on LessWrong yet.