How do you program the AI to do what humankind would approve? A superintelligent AI, perhaps even a human-level AI, would probably know what humans would approve of. The hard part is making it care about what humans think.

We can't judge based on behaviour that some one is superintelligent or not.

Yes, this is a problem. One way would be to just discard solutions that a human couldn't invent with greater than 1% probability.

Another solution would be to not have that requirement at all. Instead have it try to mimic what a human given a year to work on it, would produce. So if humans can't solve the problem, it can still show us how to make progress on it.

it doesn't optimize without end to create the best solution possible, it just has to meet some minimum threshold, then stop.

It's easy to ask hard questions. I think it can be argued that emulating a human is a hard problem. There doesn't seem to be a guarantee that the "minimum threshold" doesn't involve converting planetary volumes to computronium.

I think the same problem is present in trying to specify minimum required computing power for a task prior to prior to performing the task. It isn't obvious to me that calculating "minimum required computing power for X" is any less difficult than performing some general task X.

My main academic interests relate to the fundamentals of communication (analogous to micro economics), along with the pattern by which information and knowledge flows throughout society (like macro economics).

Until recently my focus has been on natural language, which is why I decided to learn Japanese. Without deep understanding in a second language, my endeavor to understand the process of natural-language communication (including not only words but also gestures and so on) would be hopelessly limited. I've also spent many thousands of hours constructing various artificial verbal languages for personal note-taking and linguistic experimentation.

Over the past few days, however, I've started to turn my attention to mathematics. While languages such as English, Japanese, and so forth are one-dimensional systems isomorphic to a large range of reality and constrained by the oddities of the automatic pathways we call our "natural-language hardware", my understanding is that many fields of mathematics function as more complex and precise isomorphic systems which operate in terms of brain functions more properly called "S2" or "manual". Often they transcend the 1D line of verbal language to 2D diagrammatic representations.

See this passage from Ernst Mach (1838-1916):

Language, the instrument of this communication, is itself an economical contrivance. Experiences are analysed, or broken up, into simpler and more familiar experiences, and then symbolized at some sacrifice of precision. The symbols of speech are as yet restricted in their use within national boundaries, and doubtless will long remain so. But written language is gradually being metamorphosed into an ideal universal character. It is certainly no longer a mere transcript of speech. Numerals, algebraic signs, chemical symbols, musical notes, phonetic alphabets, may be regarded as parts already formed of this universal character of the future; they are, to some extent, decidedly conceptual, and of almost general international use. The analysis of colors, physical and physiological, is already far enough advanced to render an international system of color-signs perfectly practical.

Clearly his vision of mathematics and other pencil-and-paper artificial representational systems growing and eventually combining into a single general-use international language has not come to pass in the intervening 100+ years. Mathematics has remained a specific-use tool that boasts high levels of complexity and precision within its isolated sections of thought representation and world modeling, while having extremely low coverage of the range of topic space. Humans have made huge industrial advancements, but we still fall back on the tribal device we call "words" for most of our communication attempts.

I've spent a huge number of hours designing artificial verbal-language systems which resemble natural languages except without the grammatical irregularities or folk psychology and physics, but I hold no illusion as to the point. It's a stopgap measure that I'm using to gain greater understanding of the limitations of word-based communication in an age where such systems still reign supreme. My hope for the future lies not in words, but in general-use diagrammatic or visual communication systems which include software involvement.

It would be inefficient or even irresponsible of me to attempt to make meaningful contributions within this field without possessing a solid understanding of the historical development and epistemological underpinnings of certain high-bandwidth mathematical systems. The conclusion is that it's unimportant which mathematical field I pursue at least in the beginning, provided the field is important within the context of human societal development and in engaging the material I gain a nuanced understanding of the content and a deep appreciation of how the originators created the system. Only once I develop fluency in a sufficient number of areas will I know which specific fields to consider further.

In short: I'm interested in developing a general-purpose 2D or 3D visual representational system. Attempting such an endeavor without having an appreciation for historical attempts to create non-verbal languages would be careless.

How big a deal is this? What, if anything, does it signal about when we get smarter than human AI?

Then how are new cities ever founded? How did Belmopan, Brasília, Abuja and Islamabad do it? Look at the dozens of new cities built just in Singapore during the past half century.

The OP's proposal to build a city in the middle of the desert strikes me as similar to the history of Las Vegas. What parts of it can be replicated?

I think Seattle's South Lake Union development, kickstarted by Paul Allen and Jeff Bezos, is a counter example ...

No, it's not in California. In California a city like Mountain View blocks a company like Google from building new infrastructure on it's edges.

Perhaps gentrification is a more general counter example.

In what sense? Gentrification simply means that rents go up in certain parts of the city. It doesn't have directly something to do with new investments.

It seems like you can't do incremental development by building more real estage inside the cities because of the cities not wanting to give new building permits that might lower the value of existing real estage.

It's expensive but interest rates are low and the possible profit is huge.