Fortunately, the LLN isn't just some black box. It has a proof! And we can look at that proof and get bounds on how quickly the average converges to the mean (which is basically Chebyshev's inequality, but whatever).

In cases with slightly more regularity to them, we can use Hoeffding's inequality or something similar and get even better bounds. In fact, this will give results that are almost as good as the assume-it's-normal strategy, but with the added benefit that you're actually answering the question you started with, rather than making something up.