This is pretty thought-provoking; thanks for laying it all out. I think each of the devils are in their respective details. People have very different intuitions about, for example, how many people will be turned off by a quiz requirement, or how many useful contributions would be cut off by a karma restriction on comment quantity, and it's hard to make progress toward quantifying that without running experiments which may be temporarily harmful, have confounding factors, and take a lot of manpower.

In the end, we usually settle on "loudest intuition wins" but it would be nice to make some progress on that.