I wonder whether you could explain the way of it to them and then obliviate them so that they only remember the decision to do it but not the why.

Or perhaps you could have someone who agreed to follow your orders without knowing why, specifically for the purpose of doing things with time-turners, rotate the duty through your inner circle so you don't end up with a drone.