The babyeater question would be substantially changed if the children didn't mind being eaten and didn't take harm by it - more or less from a moral crusade into parochial squeamishness. Eliezer went a long way out of his way to avoid that in the story, but here we can't dodge it with a rhetorical flourish.

If as it turns out, kids enjoy consensual sex and take no harm by it, on what basis can society consider it wrong? There has to be a reason. Societies can't just create moral crimes by their say-so.

Edit in Feb 2013: I've come to the conclusion that the problem with the above is that children are in an extremely steep power relationship - an artefact of this society, and it's avoidable, but it can't be wished away without a huge job of dismantling. Meaning, that right now children can't even express a preference. "Yes" is meaningless with the ability of an adult to apply pressure that would count as felony kidnapping and torture if done to another adult, with complete impunity and even acclaim. "No" is meaningless when adults have imposed their schemas of asexual innocence willy-nilly over children's experience, and when they have such huge control of that experience itself, up to and including maintaining "big lies" via censorship.

As such, an age of consent is a damn dirty hack that acknowledges the completely untenable position of children in making a decision that's true to their intent, while refusing to rescue them from it. It is marginally better than nothing. If it does go, it can't go first. A lot of rescuing needs to come first.