So... I'm confused. You say:
"...the half-silvered mirror rule is "Multiply by 1 when the photon goes straight, and multiply by i when the photon turns at a right angle."
We appear to have defined everything needful, except the word "when".
Accepting that we are just performing 'operations' on 'configurations', what decides which operation will be performed? Is it the configuration of the incoming photon? Is it some magical (i.e.quantum) property of a half of a silver?
This may be a repurposing of a hunting behavior - to kill an X you have to think like an X.