I'll bite the bullet. Your reasoning is correct. Assuming the laws are just and everything scales linearly, we should change our legal system to increase penalties for rarely-detected crimes.
The "everything scales linearly" is a big assumption, though. It's not very clear that deterrence scales linearly: I don't remember whether the RIAA's last suit for illegal music downloads made them ten thousand or ten million dollars, but I don't think I'd be a thousand times less likely (or even slightly less likely) to download music in the second case. If there's no difference between the deterrence value of the two cases, then society just took $9990000 worth of utility away from someone for zero gain.
And punishment doesn't scale exactly linearly either: fining 100% of a person's net worth is more than twice as bad as fining 50% of a person's net worth.
So I'm not sure I would be willing to fully endorse the new punishment scheme unless social scientists and statisticians came up with a model that adjusted for all of this in order to produce the greatest net happiness to society. But in a hypothetical situation where there was a perfect statistical model for all of this...go for it.
Also, great title :)
Cass Sunstein, David Schkade, and Daniel Kahneman, in a 1999 paper named Do People Want Optimal Deterrence, write:
If we're after optimal deterrence, we should punish potentially harmful actions more if they're hard to detect, or else the expected disutility of the punishment is too small. But apparently this does not accord with people's sense of justice.
Does this mean we should change our sense of justice? And should we apply optimal deterrence theory to informal social rewards and punishments, such as by getting angrier at antisocial behaviors that we learned of by (what the wrongdoer thought was) a freak coincidence?