My experience has been exactly contrary: young communities thrive without gardening, but as they grow they either devolve into low average value (digg as it was, most large subreddits) or are heavily pruned (HN, r/askscience). If there's an influx of people, heavy moderation is mandatory if you want to avoid regression to the mean.
If it's worth saying, but not worth its own post, even in Discussion, it goes here.