Personally, there's a certain fetish that I have, and I remember it causing me erections even before puberty. However, as far as I can recall, the experience didn't feel like anything that I'd call sexual these days. It was something that was pleasant to think about, and it caused physical reactions, but the actual sexual tension wasn't there.

I also recall a friend mentioning a pre-pubescent boy who'd had a habit of masturbating when there was snow outside, because he thought the snow was beautiful. (I'm not sure if she'd known the boy herself or if she'd heard it from someone else, so this may be an unreliable fifth-hand account.) If it was true, then it sounds (like my experience) that part of the hardware was in place, but not the parts that would make it sexual in the adult sense of the word.

Googling for "child sexuality" gives me a report from Linköping University which states on page 17:

The staff caring for 251 children aged two to six of both sexes observed the children’s behaviour and then answered a questionnaire on the behaviours they had observed. ... A total of 6% of the children had at some time been seen to masturbate and this usually occurred during rests. Masturbation took place “often/daily” in only 2% of the children. In almost every case the staff judged the masturbation to be associated with desire and relaxation on the part of the child and not in any case as painful, while one child was considered to masturbate compulsively.

It does, however, also remark that child sexual abuse often causes sexualized behavior in children, and that very little is known about what is actually normal child sexuality. Interestingly, as it relates to the original topic, it also mentions a study that found one third of abuse victims to show no symptoms at all.