Would this difference disappear if we developed the technology to turn millions of sperm cells into babies?

Probably, but in such a world, I don't think human life would be scarce, and I think that the value of human life would plummet accordingly. They would still represent a significant time and capital investment, and so be more valuable than the em case, but I think that people would be seen as much more replaceable.

It is possible that human reproduction is horrible by many moral standards which seem reasonable. I think it's more convenient to jettison those moral standards than reshape reproduction, but one could imagine a world where people were castrated / had oophorectomies to prevent gamete production, with reproduction done digitally from sequenced genomes. It does not seem obviously worse than our world, except that it seems like a lot of work for minimal benefit.