It may be that Voldemort didn't have the manpower to take out the Potters, the Death Eaters tied up elsewhere.

In Coordination Problems, Part 2

"Make no mistake," said Professor Quirrell. "The Dark Lord was winning. There were fewer and fewer Aurors who dared face him, the vigilantes who opposed him were being hunted down. One Dark Lord and perhaps fifty Death Eaters were winning against a country of thousands. That is beyond ridiculous! There are no grades low enough for me to mark that incompetence!"

In order for fifty Death Eaters to wreak the havoc that magical Britain is recovering from, they would literally need to be working round-the-clock, the Dark Mark being an example of the extreme discipline and obedience needed to be one. Voldemort may have wanted the Potters taken out ASAP, but he'd already sent everyone out on assignments. Snape wasn't going to do it, and based on what we hear about the Potters, he and Bellatrix were probably the only Death Eaters capable of actually taking them out.