Are you talking about free time pre- or post-quals? And do you include work that goes towards your thesis but that you "have" to do (e.g. for a conference or internal deadline) as free time or non-free time?
My experience (and I would guess many of my labmates, though I don't know for sure) is that quals are really easy to pass, you spend at most 2 weeks of your life studying for them, and otherwise you're just doing research plus a few classes. Stanford is an outlier in that it has particularly few class requirements compared to other top CS departments, but it seemed like MIT grad students also often started doing research fairly early on, from my perspective as an undergrad there.
Depending on your funding situation, your actual time spent doing research may be more or less beholden to what grants your advisor has to do work towards. I'm on a fellowship and so can do whatever I want, the only consequences being that if my research after 5 years is uninteresting then I'll have trouble getting academic jobs.
At the U. of Buffalo, just taking the quals took at least a week. They were, if I recall, 7 exams, 5 taking half a day each and 2 24-hour exams.
Among my friends interested in rationality, effective altruism, and existential risk reduction, I often hear: "If you want to have a real positive impact on the world, grad school is a waste of time. It's better to use deliberate practice to learn whatever you need instead of working within the confines of an institution."
While I'd agree that grad school will not make you do good for the world, if you're a self-driven person who can spend time in a PhD program deliberately acquiring skills and connections for making a positive difference, I think you can make grad school a highly productive path, perhaps more so than many alternatives. In this post, I want to share some advice that I've been repeating a lot lately for how to do this:
That's all I have for now. The main sentiment behind most of this, I think, is that you have to be deliberate to get the most out of a PhD program, rather than passively expecting it to make you into anything in particular. Grad school still isn't for everyone, and far from it. But if you were seriously considering it at some point, and "do something more useful" felt like a compelling reason not to go, be sure to first consider the most useful version of grad that you could reliably make for yourself... and then decide whether or not to do it.
Please email me (lastname@thisdomain.com) if you have more ideas for getting the most out of grad school!