Historically, the usual problem with that is that empowering competent people to make political decisions also empowers them to decide the meaning of "competent", and the meritocracy slowly turns into an aristocracy. The purest example I can think of offhand is the civil service examinations gating bureaucratic positions in imperial China, although that wasn't a democratic system.