Yes it would require a lot of human input.

However the AI could learn to predict what humans like, and then use that as it's judge. Trying to produce songs that it predicts humans will like. Then when it tests it on actual humans, it can see if it's predictions were right and improve them.

This is also a domain with vast amounts of unsupervised data available. We've created millions of songs, which it can learn from. Out of the space of all possible sounds, we've decided that this tiny subset is pleasing to listen to. There's a lot of information in that.