Attempts have been made to reduce the Big Five into a "Big One", or "General Factor of Personality"(GFP), this correlates the way you describe it. The neurotism is sometimes called Stability, and this together with the other four correlate with one another. Here's a paper by Rushton et al:

A recent observation is that a General Factor of Personality (GFP) occupies the apex of the personality hierarchy in the same way that g, the general factor of mental ability, occupies the apex in the organization of cognitive abilities. Individuals high on the GFP are characterized as altruistic, emotionally stable, agreeable, conscientious, extraverted, intellectually open, mentally tough, and emotionally intelligent; ...

(I have low GFP: I'm rather miserable...)