Vocal articulation is hard, requiring the simultaneous coordination of glottis, velum, tongue, and mouth, not to mention airflow. Hands are bigger, the movements don't need to be as precise, and some of the signs partially resemble the thing/action they represent.