extremely powerful guard unions
This doesn't sound right. Russia's prisons are pretty horrible too, but we don't have powerful guard unions.
I'm inclined to agree with you-- the prison-industrial complex affects the number of people in prison in the US, but it's not the only way that sort of thing can happen.
As for the US, it's not just the guards' unions, though the unions have lobbied for "tough on crime" measures-- there's money in constructing prisons and in for-profit prisons. Also, prisons can be a major source of employment in rural areas, and prisoners count as residents for counties (I think-- it might be at some other organizational level) to get Federal aid, but the money ...
Why do those words go together?
Society - and for once, I'm using this term universally - teaches that, if you committed a crime, you should be punished.
But in some societies, we have an insanity defense. If you had a brain condition so that you had no - here it's a little vague - consciousness, or moral sense, or free will, or, well, something - then it would be cruel to punish you for your crime. Instead of going to prison, you should be placed somewhere where you can't hurt anybody, where professional physicians and counselors can study your case and try to reform you so that you can rejoin society.
Wait - so that isn't what prison is for?
No. Prison is to punish people. Is it any wonder that prison doesn't reform people, when we don't want it to reform them? Most people would be upset if prisoners could go in on Friday, and emerge, rehabilitated, on Sunday. When people say, "It would be cruel to punish people who aren't responsible for their actions", they are implicitly saying, "Prison is necessarily cruel; and that's good, because we should be cruel to criminals who are responsible for their actions."
But the more we learn about psychology and neuroscience, the further responsibility recedes into the distance.
Outcome-based justice argues that we should give up playing the blame game, because neuroscience keeps finding more and more proofs that things are "not our fault". Instead, we should write laws that deter crime.
You might think this is what we already try to do. But it isn't! Witness this confused article from the Brookings Institute, Cognitive Neuroscience and the Future of Punishment by O. Carter Snead. Snead objects to outcome-based justice. He summarized all of the arguments for it, yet managed to completely miss their point, concluding where he started from, saying that outcome-based justice is obviously bad because it could lead to being cruel to people who didn't deserve it. (Instead of only being cruel to the people who do deserve it, which is obviously what we want to do.)
Snead understands that outcome-based justice deters crime:
You might expect that Snead goes on to explain why these laws are bad things. But he doesn't! He assumes we can all see that these are obviously bad things.
The Wrath of Kahneman describes a study which asked whether people punish others in order to deter crime, and concluded, No. People are doing something else.
One theory is that people are trying to be fair. Everyone should get the same chances; everyone should get the same punishment for the same crime. John Rawls argues this explicitly in Justice as Fairness: Justice should not be utilitarian, but should instead be fair.
I believe Rawls' view is also the popular view of what "justice" means. And, I will argue in a later post, it is part of a pattern showing a deep divide between two different ways of using the word "ethics".
ADDED: Constant made the point that, while one part of outcome-based justice is preventing future harm from the criminal on the dock, another part is deterring harm by other criminals. This latter part does not benefit from punishing criminals who cannot be deterred. Thus, to optimally punish both criminals who can and cannot be deterred, the law requires a concept of moral culpability, and should punish criminals who can't be deterred more lightly. This is a better origin story for the linking together of morality and free will than the just-so story I'd come up with, so I plan on stealing it for my next post. (SilasBarta may have been trying to make the same point, but I found his comments impenetrable.)
(This post is laying groundwork for two other posts that will go in different directions, neither of which concerns justice.)