Yup, that's what makes it quite possible. Not drop-dead obvious, though - there's nonzero selection pressure for intelligence even now, environment has a huge load to do with intelligence so there are possibly big gains between the beginning to civilization and now, maybe civilization-making magic happens with enough geniuses (genii?) or is prevented by too many too stupid people so variance (and maybe population) matters more than average.
Link: abcnews.go.com/Health/friendly-extreme-meet-kids-adults-williams-syndrome/