He also praised the idea of encouraging spies in certain departments rather than prosecuting them, in order to keep the Soviet Union from worrying that we were planning a first strike.

In general, a caught spy can be a useful resource. If you've identified a spy but the spy doesn't know he's been found out, you get to control what the spy's reports are.