The TDT-style reasoning is not "would this be bad if everyone did it?" but "would this be bad if everyone sufficiently similar to me did it?" and I think there it's much less clear. If everyone similar to me spent less time on politics and more time on more useful things I don't think that's obviously a net loss at all.
To be slightly more precise, the TDT-style reasoning is, "Would this be bad if everyone who decided using a decision procedure logically correlated with mine did it?".
Now, it might be that your decision procedure is so "logically isolated" that the cohort of people whose decisions are correlated with yours is too small to be politically significant.
But it seems to me that most people arrive at their political opinions using one of a small set of correlated classes of decision procedures. It follows from the pigeon-hole principle that at least one of these correlated classes contains a lot of people. (The fact that you can write above about how "most people approach politics" also points towards this conclusion.) There is a real chance that this class is large enough to be politically significant.
The upshot is that your reasoning, while it might apply to you, would not apply to people who decide these issues in more typical ways, because these people are numerous enough that their political opinions have real influence.
Which is not to say that these people need to be spending more time on politics. But it does suggest that their getting their politics right matters.
So you're playing the credence game, and you’re getting a pretty good sense of which level of confidence to assign to your beliefs. Later, when you’re discussing politics, you wonder how you can calibrate your political beliefs as well (beliefs of the form "policy X will result in outcome Y"). Here there's no easy way to assess whether a belief is true or false, in contrast to the trivia questions in the credence game. Moreover, it’s very easy to become mindkilled by politics. What do you do?
In the credence game, you get direct feedback that allows you to learn about your internal proxies for credence, i.e., emotional and heuristic cues about how much to trust yourself. With political beliefs, however, there is no such feedback. One workaround would be to assign high confidence only to beliefs for which you have read n academic papers on the subject. For example, only assign 90% confidence if you've read ten academic papers.
To account for mindkilling, use a second criterion: assign high confidence only to beliefs for which you are ideologically Turing-capable (i.e., able to pass an ideological Turing test). As a proxy for an actual ideological Turing test, you should be able to accurately restate your opponent’s position, or be able to state the strongest counterargument to your position.
In sum, to calibrate your political beliefs, only assign high confidence to beliefs which satisfy extremely demanding epistemic standards.