Well the main problem is the sheer severity of the Doomsday effect.

Suppose space-colonizing civilizations have an average population a billion times that of "doomed" civilizations, then the mediocrity argument implies that fewer than 1 in a billion civilizations become space-colonizing. If the population ratio is a trillion, then fewer than 1 in a trillion become space-colonizing.

But there are something like 10^22 stars in the observable universe, and a space-colonizing civilization could reach a very large portion of them; further, it would tend to do so, if there is no real competition from other colonizing civilizations (the competition would instead be arising at the edge of the expansion wave, causing travel speeds to increase and approach the speed of light). So the most likely population increase factor is something like a billion trillion or more, implying a chance of civilization survival of 1 in a billion trillion or less. That does seem unreasonably pessimistic.