Given a finite graph you can define a characteristic of that graph that for our purposes is called "modularity."

For all integers N, consider all of the ways that you can define a partition of the graph into N subsets. For each partition, divide the size of the smallest partition by the number of connections between the different components. Find the partition where this number is maximal, then this maximal ratio is the "N-modularity" of the graph. That is, if the number is very high, there is a partition into N blocks which are dense and have few connections between each other; we can call these modules.

Not sure about how to define nested, but I imagine it has to do with isomorphic sub-graphs; so if each of the N modules of a graph had the same structure, the graph would be nested as well as modular.

But I'm less confident about the nested definition.

Modular and nested are not opposites. "Nested", they say, means a sharing of relationships; they're not any more specific than that. Don't know what you mean about being rotated by 180 degrees. Consider the lower-left picture: It shows modularity in mutualistic (cooperative) relationships. All the points are below the line y=x because the initial measure of modularity was larger than the equilibrium measure of modularity.