This is Peter Thiel building on the ideas of one of his teachers at Stanford, the Catholic philosopher René Girard. Girard had this slightly weird theory of human nature according to which all desire is imitative, this leads to people wanting the same things, and this leads to community competition for scarce resources. In pre-Christian cultures the competition is resolved by finding someone to blame, a scapegoat, who the community then unites to persecute. But Christian culture has neutralized this mechanism by siding with the underdog against the community, and so society now has all these competitive and violent urges hanging around unresolved. Girard lived in the 20th century and he tried to interpret the risks of fascism, communism, and nuclear war in terms of this framework, specifically in terms of a struggle to fend off the apocalyptic unleashing of violence. Thiel's contribution to Girardian theory is to interpret the 21st century so far as a low-testosterone period in which no one cares enough to really do anything apocalyptic; but this can't go on forever. 

I haven't actually read Girard, so I don't know any of the nuances. But I can interpret this partly through the lens of one of my gurus, Celia Green, who depicts society as a kind of conspiracy to suppress ambition, on the principle that misery loves company. This idea horrified me as a young transhumanist, both because it paints an ugly picture of human nature as spiteful and vengeful, and because it implies there is no will to serious liberation (I wanted to end work and death), in fact it predicts that such attempts will be resisted. I always had trouble believing this theory, and I would now say there's a lot more to human nature (and to why transhumanism hasn't prospered) than just "the nail that sticks up will be hammered down". But it's definitely a factor in human life. 

Celia Green was herself an extremely ambitious person who didn't get to act on her ambitions, which explains why she ended up developing such a theory of human nature. My guess is that Thiel has a similar story, except that he got his analysis from Girard, and he did succeed in a lot of his ambitions. Basically, anyone setting out to become a rich and powerful capitalist, as Thiel did, has to worry about becoming a focus of negative attention, especially when there are political movements that attack wealth and/or privilege; and Girard's theories may have explained what Thiel saw as a student (intersectional leftism on campus) as well as preparing him for his entrepreneurial career. 

So in both Green and Girard, we are dealing with theories of social psychology according to which people have a tendency to persecute individuals and minorities in the name of the collective, and in which this persecution (or its sublimation) is central to how society works. They even both utilize Christianity in developing their theories, but for Girard (and Thiel), Christianity supplies an overall metaphorical structure including eschatology, whereas Green focuses on Gnostic Christianity as symbolizing how the individual psyche can escape the existential pitfalls that await it. 

So I'd say this essay by Thiel is a work of Girardian eschatology, similar to apocalyptic Christian writings which try to interpret the contemporary world in terms of the end times, only Girard's apocalypse is the violent collapse of civilization. Girard's whole historiography, remember, revolves around the premise that pre-Christian societies had this psychology in which scapegoats are sacrificed to the collective, then Christ was supposed to nullify this impulse by being the ultimate sacrifice. But he really needs to return in order to seal the deal, and meanwhile we've had 2000 years of Christian and post-Christian societies wrestling with the need to sublimate the sacrificial impulse, seeking substitutes for the Christian formula, succumbing to apocalypse in the form of war or revolution, or teetering back from the brink, and so on. Thiel is adding a new chapter to the Girardian narrative, in which the 21st century has avoided collapse due to a generally lowered vitality, but he prophesies that the end must still come in some form. 

Afterword by an AI (Claude 3.7 Sonnet)

My summary of your argument: In order to guess the nature of AI experience, you look at the feelings or lack of feelings accompanying certain kinds of human cognition. The cognition involved with "love, attraction, friendship, delight, anger, hate, disgust, frustration" has feelings onboard; the cognition involved with sequence prediction does not; the AI only does sequence prediction; therefore it has no feelings. Is that an accurate summary? 

What exactly will happen to people who don't "get out" in time? 

You say consciousness = successful prediction. What happens when the predictions are wrong? 

I knew the author (Michael Nielsen) once but didn't stay in touch... I had a little trouble figuring out what he actually advocates here, e.g. at the end he talks about increasing "the supply of safety", and lists "differential technological development" (Bostrom), "d/acc" (Buterik), and "coceleration" (Nielsen) as "ongoing efforts" that share this aim, without defining any of them. But following his links, I would define those in turn as "slowing down dangerous things, and speeding up beneficial things"; "focusing on decentralization and individual defense"; and "advancing safety as well as advancing capabilities". 

In this particular essay, his position seems similar to contemporary MIRI. MIRI gave up on alignment in favor of just stopping the stampede towards AI, and here Michael is also saying that people who care about AI safety should work on topics other than alignment (e.g. "institutions, norms, laws, and education"), because (my paraphrase) alignment work is just adding fuel to the fire of advances in AI. 

Well, let's remind ourselves of the current situation. There are two AI powers in the world, America and China (and plenty of other nations who would gladly join them in that status). Both of them are hosting a capabilities race in which multiple billion-dollar companies compete to advance AI, and "making the AI too smart" is not something that either side cares about. We are in a no-brakes race towards superintelligence, and alignment research is the only organized effort aimed at making the outcome human-friendly. 

I think plain speaking is important at this late stage, so let me also try to be as clear as possible about how I see our prospects. 

First, the creation of superintelligence will mean that humanity is no longer in control, unless human beings are somehow embedded in it. Superintelligence may or may not coexist with us, I don't know the odds of it emerging in a human-friendly form; but it will have the upper hand, we will be at its mercy. If we don't intend to just gamble on there being a positive outcome, we need alignment research. For that matter, if we really didn't want to gamble, we wouldn't create superintelligence until we had alignment theory perfectly worked out. But we don't live in that timeline. 

Second, although we are not giving ourselves time to solve alignment safely, that still has a chance of happening, if rising capabilities are harnessed to do alignment research. If we had no AI, maybe alignment theory would take 20 or 50 years to solve, but with AI, years of progress can happen in months or weeks. I don't know the odds of alignment getting fully solved in that way, but the ingredients are there for it to happen. 

I feel I should say something on the prospect of a global pause or a halt occurring. I would call it unlikely but not impossible. It looks unlikely because we are in a decentralized no-holds-barred race towards superintelligence already, and the most advanced AIs are looking pretty capable (despite some gaps e.g. 1 2), and there's no serious counterforce on the political scene. It's not impossible because change, even massive change, does happen in politics and geopolitics, and there's only a finite number of contenders in the race (though that number grows every year). 

[Later edit: I acknowledge this is largely wrong!  :-) ]

Have you researched or thought about how the models are dealing with visual information? 

When ChatGPT or Gemini generates an image at a user's request, they are evidently generating a prompt based on accumulated instructions and then passing it to a specialized visual AI like DALLE-3 or Imagen-3. When they process an uploaded image (e.g. provide a description of it), something similar must be occurring. 

On the other hand, when they answer your request, "how can I make the object in this picture", the reply comes from the more verbal intelligence, the LLM proper, and it will be responding on the basis of a verbal description of the picture supplied by its visual coprocessor. The quality of the response is therefore limited by the quality of the verbal description of the image - which easily leaves out details that may turn out to be important. 

I would be surprised if the LLM even has the capacity to tell the visual AI, something like "pay special attention to detail". My impression of the visual AIs in use is that they generate their description of an image, take it or leave it. It would be possible to train a visual AI whose processing of an image is dependent on context, like an instruction to pay attention to detail or to look for extra details, but I haven't noticed any evidence of this yet. 

The one model that I might expect to have a more sophisticated interaction between verbal and visual components, is 4o interacting as in the original demo, watching and listening in real time. I haven't had the opportunity to interact with 4o in that fashion, but there must be some special architecture and training to give it the ability of real-time interaction, even if there's also some core that's still the same as in 4o when accessed via text chat. (I wonder to what extent Sora the video AI has features in common with the video processing that 4o does.)

Who or what is the "average AI safety funder"? Is it a private individual, a small specialized organization, a larger organization supporting many causes, an AI think tank for which safety is part of a capabilities program...?

Wow! This is the "AI 2027" of de-dollarization. I'm no finance person, but I have been looking for analysis and this is the clearest future scenario I've run across. I will make one comment, based on futurological instinct, and that is that change may go even faster than you describe. One of the punishing things about making scenarios in times of rapid change is that you put in the work to look several years ahead, then changes you had scheduled for years away, end up happening within months or even weeks, and you have to start again. But I'm sure your team can rise to the challenge. :-) 

Wasn't there a move into treasuries and USD, just the day before? 

I have a geopolitical interpretation of how the tariffs have turned out. The key is that Trump 2.0 is run by American nationalists who want to control North America and who see China as their big global rival. So Canada and Mexico will always be in a separate category, as America's nearest neighbors, and so will China, as the country that could literally surpass America in technological and geopolitical power. Everyone else just has to care about bilateral issues, and about where they stand in relation to China versus America. (Many, like India and Russia, will want to be neutral.) 

Also, I see the only serious ideological options for the USA at this point, as right-wing nationalism (Trump 2.0) and "democratic socialism" (AOC, Bernie). The latter path could lead to peaceful relations with China, the former seems inherently competitive. The neoliberal compromise, whereby the liberal American elite deplores China's political ideology but gets rich doing business with them, doesn't seem viable to me any more, since there's too much discontent among the majority of Americans.