Opportunity costs, in short. If you're giving up more resource-equivalent time on that nanoslice of power than you expect it to return in dividends, it's not worth your effort -- and depending on how you do the counting, a lot of prominent examples return so little that it doesn't take much time outlay for this to be the case.

In the specific case of voting, though, there are signaling effects to consider that might overwhelm its conventional dividends. Jurisdictions like Australia where voting is mandatory also change the incentive landscape.