I think anything that starts to go down the 'intelligent/educated have more power' route is missing the most important elements of democracy. Not only are you cutting out large parts of the population, but these would inevitably have locality/race/income correlations that made the whole thing even uglier. Probably even gender correlations.
Not to mention the problem that people have to choose to take it, and I'm not convinced that wanting to have that power is a good sign that you should. You might have a disproportionate number of ideologues/slight megalomaniacs. Not to mention the 'tragedy of the commons' issue mentioned elsewhere.
The budget stalemate in the US Congress was caused entirely by blocks of voters and representatives that coalesced around strong sets of opinions that few people would have come up with on their own, and by political party leaders forcing representatives in their parties to toe the party line. Politics isn't the mind killer. Political parties are the mind-killer.
Parties are also notorious for obliterating information in elections, as well as for encouraging voters to vote sans information. If you went to your polling place and saw a list of candidates, none of whom you'd heard of before, you might rightly refrain from voting and polluting the signal with your noise. Knowing party affiliations makes people think they have enough information to vote.
For discussion:
We want the freedom to form groups that promote political concerns. But it would be possible to keep these groups at a greater distance from elected representatives. Candidates for office could be forbidden from endorsing a particular party. The Congress could be forbidden from basing any procedural rules on party affiliation. Political parties could be forbidden from making large donations to election campaigns, or sponsoring advertising. That's not so different from what we do today with religious groups, which are not much different from political parties.
Political parties are currently officially part of Congress' operation, even though they're not in the constitution. There are all sorts of Congressional rules specifying how the parties interact, who gets to choose committee members, who runs the House and Senate floors, etc. A party leader can punish a representative who doesn't toe the line with many incentives and disincentives.
Make that illegal. Make persecuting a representative for party-based reasons have the same legal standing as persecuting a representative for religious reasons.
I will ignore comments saying "you're an intellectual dreamer", for the usual reasons.