I have seen it suggested that what people think of when they hear "a life barely worth living" is a live on the edge of suicide, which is well past not worth living. As such, it's not surprising that you wouldn't want a world full of people like that.

If the lives are worth living, then it seems to me intuitively obvious that sufficiently many can be arbitrarily valuable.