My guess is just that the original reason was that there were societal hierarchies pretty much everywhere in the past, and they wanted some way to have nobles/high-status people join the army and be obviously distinguished from the general population, and to make it impossible to be demoted far down enough so as to be on the same level. Armies without the officer/non-officer distinction just didn't get any buy-in from the ruling class, and so they wouldn't exist.

I think there's also a pretty large difference in training -- becoming an officer isn't just about skills in war, but also involves socialization to the officer culture, through the different War Colleges and whatnot.