Singer's position is worth quoting at length (emphasis mine):

I use the term "person" to refer to a being who is capable of anticipating the future, of having wants and desires for the future. As I have said in answer to the previous question, I think that it is generally a greater wrong to kill such a being than it is to kill a being that has no sense of existing over time. Newborn human babies have no sense of their own existence over time. So killing a newborn baby is never equivalent to killing a person, that is, a being who wants to go on living. That doesn’t mean that it is not almost always a terrible thing to do. It is, but that is because most infants are loved and cherished by their parents, and to kill an infant is usually to do a great wrong to its parents. Sometimes, perhaps because the baby has a serious disability, parents think it better that their newborn infant should die. Many doctors will accept their wishes, to the extent of not giving the baby life-supporting medical treatment. That will often ensure that the baby dies. My view is different from this, only to the extent that if a decision is taken, by the parents and doctors, that it is better that a baby should die, I believe it should be possible to carry out that decision, not only by withholding or withdrawing life-support – which can lead to the baby dying slowly from dehydration or from an infection - but also by taking active steps to end the baby’s life swiftly and humanely.

Most parents, fortunately, love their children and would be horrified by the idea of killing it. And that’s a good thing, of course. We want to encourage parents to care for their children, and help them to do so. Moreover, although a normal newborn baby has no sense of the future, and therefore is not a person, that does not mean that it is all right to kill such a baby. It only means that the wrong done to the infant is not as great as the wrong that would be done to a person who was killed. But in our society there are many couples who would be very happy to love and care for that child. Hence even if the parents do not want their own child, it would be wrong to kill it.

Singer's position seems to be pretty much exactly mine with one important exception. He thinks that the reason infanticide in general is wrong is because killing someone else's baby is a great wrong to the parent. This I fully agree with and is why I have repeatedly specified that you should only be able to kill your own child. He further thinks that killing any animal is wrong if it could have a happy life if you chose not to kill it, and for this reason he thinks killing infants is wrong. This second position I disagree with, but I suspect almost anyone here who is not a vegetarian also disagrees. daenerys seems to have come to much the same conclusion as Singer.

ETA: I'm having trouble finding philosophers apart from Singer and Tooley who have written on this topic at all, and both seem to have come to roughly the same conclusions that I did. This is kind of unfortunate, because I'm definitely interested in what the other side is - but as things stand, the most strong arguments against infanticide I've seen have already been presented in this thread and have to do with practical concerns, like the risk of increasing sadistic behaviors and the importance of using Schelling points where practical.

ETA2: Here's another paper, which directly addresses Tooley. I can temporarily post it somewhere if people don't have access to JSTOR. The gist of the argument is that Tooley needs to make a consequentialist argument, not just a moral one. The conclusion of the authors is that they don't think such an argument would be hard to make but that Tooley definitely failed to make it. (Incidentally, as far as I can tell and as Multiheaded points out, Singer also fails to spend much consideration on the full societal consequences of legalized infanticide.)