I think it's pretty clear that animals can feel pain, distress, etc. So we should aim for practices that minimise those things. It's certainly possible - though harder on a mass scale like factory farming.

Also, from a utilitarian perspective, it's clear that eating plants is much more ecologically efficient than feeding plants to animals and then eating the animals. On the other hand, as Elo points out, there are crops and terrain that are not well suited to human food, and might more profitably be used to raise edible animals.

So I'd say that there could be an equilibrium, a point where our overall meat consumption is about right; less would be basically a wasted opportunity; more would be an inefficient use of resources and a risk of oppressive practices. And I'd say that that point is much lower than current overall consumption.