I'd be interested in any ideas as to what makes this mental state different, and how to cultivate it.

Usually just being tired and stopping myself from falling asleep is enough. But nothing I hear this way can be written down, as if it's another type of music altogether.

I find it's easier to imagine novel music if I restrict myself to a particular genre+style. Just settling on the instruments helps a lot.

I've found the same thing with any kind of composition. And with many kinds of problems, adding constraints (e.g. choosing an instrument or form in music, choosing rhyme or meter in poetry), even arbitrary constraints, actually makes them easier to solve. More on this in the future.