But remember, Von Neumannn was pronounced, by a peer, to be smarter than Albert Einstein to his face and got no objection.
This is talking about intelligence, which I think von Neumann did appreciate, but he was worried about his overall legacy or achievement, due to lack of sufficient creativity. See this passage from Wikipedia:
Ulam suggests that some of his self-doubts with regard for his own creativity may have come from the fact he had not discovered several important ideas that others had, even though he was more than capable of doing so, giving the incompleteness theorems and Birkhoff’spointwise ergodic theorem as examples. Von Neumann had a virtuosity in following complicated reasoning and had supreme insights, yet he perhaps felt he did not have the gift for seemingly irrational proofs and theorems or intuitive insights.
So instead of "he never appreciated his own capabilities" I would say he actually exhibited an impressive level of metacognition and self-awareness.
A report on this question produced by AI (Kimi K2 research mode):
The evidence overwhelmingly supports the conclusion that John von Neumann was indeed profoundly insecure about his intellectual legacy, despite his monumental achievements across multiple fields of science and mathematics.
The testimonies of his daughter Marina von Neumann Whitman and his close friend Stanisław Ulam provide direct, first-hand accounts of his persistent anxiety about whether his work would endure. These are not speculative interpretations but documented observations from the people who knew him best.
See also the Self-doubts section of his Wikipedia page:
Rota wrote that von Neumann had "deep-seated and recurring self-doubts".[394] John L. Kelley reminisced in 1989 that "Johnny von Neumann has said that he will be forgotten while Kurt Gödel is remembered with Pythagoras, but the rest of us viewed Johnny with awe."[395] Ulam suggests that some of his self-doubts with regard for his own creativity may have come from the fact he had not discovered several important ideas that others had, even though he was more than capable of doing so, giving the incompleteness theorems and Birkhoff's pointwise ergodic theorem as examples. Von Neumann had a virtuosity in following complicated reasoning and had supreme insights, yet he perhaps felt he did not have the gift for seemingly irrational proofs and theorems or intuitive insights. Ulam describes how during one of his stays at Princeton while von Neumann was working on rings of operators, continuous geometries and quantum logic he felt that von Neumann was not convinced of the importance of his work, and only when finding some ingenious technical trick or new approach did he take some pleasure in it.[396] However, according to Rota, von Neumann still had an "incomparably stronger technique" compared to his friend, despite describing Ulam as the more creative mathematician.[394]
6A few more related thoughts:
- My dilemma is also a real-world counter-example/analogy for Strategy Stealing: if a misaligned AI does something that's unethical from my perspective to benefit itself, how am I supposed to copy its strategy?
- An alternative hypothesis I considered was that Anthropic was looking for oversight from or accountability to x-safety-conscious people, but IIRC the investment was structured through an SPV and people like me would have no voting rights, which would instead be held by the SPV's manager (who was not known as someone very concerned about AI x-safety). Their explanation was that this is common in tech startups, which I believe is technically correct, but obviously did nothing to make me less worried about their safety/governance views given that alternatives like pass-through voting are also available and sometimes used.
- I always thought it was totally crazy for people to lump Nick Bostrom and Marc Andreessen together into TESCREAL and criticize them in the same breath, but this post plays right into such criticism. (This is one of the "other political or ethical perspectives" I alluded to.) Maybe it is still wrong or unfair, but given the apparent alignment between the OP's position and Andreessen's interests, I would have upgraded such criticism from "totally crazy" to "worth addressing". (I'm also forced to mentally assign some credit to such critics for apparently recognizing or predicting such alignment, that I'm personally surprised by, and which now undeniably exists at least at a surface level.)
I passed up an invitation to invest in Anthropic in the initial round which valued it at $1B (it's now planning a round at $170B valuation), to avoid contributing to x-risk. (I didn't want to signal that starting another AI lab was a good idea from a x-safety perspective, or that I thought Anthropic's key people were likely to be careful enough about AI safety. Anthropic had invited a number of rationalist/EA people to invest, apparently to gain such implicit endorsements.)
This idea/plan seems to legitimize giving founders and early investors of AGI companies extra influence on or ownership of the universe (or just extremely high financial returns, if they were to voluntarily sell some shares to the public as envisioned here), which is hard for me to stomach from a fairness or incentives perspective, given that I think such people made negative contributions to our civilizational trajectory by increasing x-risk.
I suspect that others will have other reasons (from other political or ethical perspectives) to object to granting or legitimizing a huge windfall to this small group of people, and it seems amiss that the post/paper is silent on the topic.
10
2@Lukas Finnveden I originally heard it in a Chinese language YouTube video about Hu Shih. I tried to look up a reference before making the post, but couldn't find anything online that was well-sourced so decided not to cite anything, but this is the best that I can find, with English translation below:
AI translated article about Hu Shih's marriage
The Marriage of Hu Shih and Jiang Dongxiu: One of the Great Oddities of the Republican Era
"You want a divorce? Fine. Since you don't want us anymore, I'll kill our two sons first, then kill myself right in front of you!" Jiang Dongxiu said, full of grief and indignation.
Hu Shih looked in horror at the kitchen knife held to the neck of his 5-year-old son, Liwang, and at his 2-year-old son, Sidu, in his wife's arms. Terrified, he fled. All thoughts of divorce could go to hell.
The woman holding the kitchen knife was Jiang Dongxiu, the arranged-marriage wife of Hu Shih, a leader of the New Culture Movement and a great literary master. Their marriage was known as one of the "Great Oddities of the Republican Era."
The pairing of a cultural elite with an uneducated village woman was not surprising in itself; Lu Xun, Xu Zhimo, and Yu Dafu all abandoned their first wives. What seemed strange was that Hu Shih never abandoned Jiang Dongxiu.
Was Jiang Dongxiu Hu Shih's true love? No. Hu Shih agreed to marry Jiang Dongxiu entirely to please his mother. Thus, the thirteen-year-old boy and the fourteen-year-old girl were engaged. Afterwards, Hu Shih began his thirteen-year journey of study in Shanghai and the United States, while the yet-to-be-wed Jiang Dongxiu moved into the Hu family home to care for his mother.
During his studies in America, Hu Shih met the beautiful and intelligent American woman, Edith Clifford Williams.
["Although in the past forty-eight hours, I have already written you two letters and a postcard, I can't resist writing another letter..."]
From excerpts of their letters, one can see the fiery passion in their relationship. During this same period, Hu Shih's views on women's education also changed:
The purpose of women's education is not to prepare them to be good wives and mothers, but to cultivate free and independent women.
This shift in thinking guided his actions. Across the ocean, Jiang Dongxiu received a letter from Hu Shih asking to break off the engagement.
Hu Shih's formidable mother said bluntly, "This is absolutely impossible, get that idea out of your head!" At the same time, she wrote a letter to Edith's mother, informing her of the fact that Hu Shih already had a fiancée. In the end, this transnational romance was crushed by the iron will of the two matriarchs.
In 1917, the 27-year-old Hu Shih and the 28-year-old Jiang Dongxiu were finally married. Who could have known that behind this seemingly happy ending was the beginning of a nightmare.
At the wedding, the person who caught Hu Shih's eye was not his bride, but the bridesmaid, Cao Chengying, who called him "Brother Mi."
Cao Chengying loved flowers and literature. In her letters to Hu Shih's wife, "Sister-in-law Dongxiu," she would enclose poems for "Brother Mi" to appreciate, which Hu Shih was always delighted to do.
The affair between them began during a visit. When Hu Shih was recuperating from an illness in Hangzhou, Cao Chengying, who worked there, went to visit him. The two toured Hangzhou together for four days. As they parted, he gave her a short vernacular poem titled "West Lake."
The West Lake I dreamed of for seventeen years,
Cannot cure my illness,
But instead makes it worse
...
Now that I've returned,
I only find it more lovely,
And thus cannot bear to leave so soon.
A few days later, Hu Shih secretly traveled from Shanghai to the shores of West Lake in Hangzhou, rented a few rooms in a temple, and lived with Cao Chengying for three months. Afterwards, he went home and formally asked his wife for a divorce, to which Jiang Dongxiu gave her forceful response with a kitchen knife.
From then on, Hu Shih never mentioned divorce again and became the model of a henpecked husband in literary circles. He even wrote the "New Three Obediences and Four Virtues Song":
[The Three Obediences: One must obey when the wife gives an order; one must follow along when the wife goes shopping or plays mahjong; one must blindly obey when the wife loses her temper for the wrong reason. The Four Virtues: One must be willing to spend when the wife buys things; one must be able to endure when the wife gets angry; one must remember the wife's birthday; one must be able to wait when the wife is dressing up to go out.]
As for Cao Chengying, she could only grieve in sorrow and had to abort the child she was carrying.
Can a relationship last long based only on threats and intimidation? A kitchen knife can kill, but it can also express tenderness.
Jiang Dongxiu could use a kitchen knife to defend her marriage, but she could also cook Hu Shih's favorite Anhui dish, "Yipin Guo." She provided meticulous care in their daily life, an all-powerful homemaker who never let Hu Shih worry about a thing. In her interactions with her husband, she would even speak straightforward words of love:
["Mr. Gao said you can't live in Shanghai anymore, he said your health this time is not as good as before. Today I heard him say you are not very well, and my heart felt like it was being cut by a knife. No matter what, I beg you, as soon as you see my letter, to hurry back to Beijing. I have been sick for three days..."]
This was a letter Hu Shih received from his wife while he was ill. Although it was filled with misspelled words, it delighted him, and he even wrote a poem in response:
Sick in bed, I get a letter from my wife,
Not even eight full lines of paper;
Nothing of importance in it,
Yet it makes me quite happy.
In this back-and-forth, sweetness was found between the lines. The village woman with bound feet, Jiang Dongxiu, used both hard and soft tactics to manage the great literary master Hu Shih, securing her own position and growing old together with him.
Some quotes from it:
A few days later, Hu Shih secretly traveled from Shanghai to the shores of West Lake in Hangzhou, rented a few rooms in a temple, and lived with Cao Chengying for three months. Afterwards, he went home and formally asked his wife for a divorce, to which Jiang Dongxiu gave her forceful response with a kitchen knife.
From then on, Hu Shih never mentioned divorce again and became the model of a henpecked husband in literary circles. He even wrote the "New Three Obediences and Four Virtues Song":
[The Three Obediences: One must obey when the wife gives an order; one must follow along when the wife goes shopping or plays mahjong; one must blindly obey when the wife loses her temper for the wrong reason. The Four Virtues: One must be willing to spend when the wife buys things; one must be able to endure when the wife gets angry; one must remember the wife's birthday; one must be able to wait when the wife is dressing up to go out.]
Prestige status is surprisingly useless in domestic life, and dominance status is surprisingly often held by the female side, even in traditional "patriarchal" societies.
Examples: Hu Shih, the foremost intellectual of 1920s China (Columbia PhD, professor of Peking University and later its president), being "afraid" of his illiterate, foot-bound wife and generally deferring to her. Robin Hanson's wife vetoing his decision to sell stocks ahead of COVID, and generally not trusting him to trade on their shared assets.
Not really sure why or how to think about this, but thought I'd write down this observation... well a couple of thoughts:
- Granting or recognizing someone's prestige may be a highly strategic (albeit often subconscious) decision, not something you just do automatically.
- These men could probably win more dominance status in their marriages if they tried hard, but perhaps decided their time and effort was better spent to gain prestige outside. (Reminds me of comparative advantage in international trade, except in this case you can't actually trade the prestige for dominance.)
1AI written account of how Mao came to make the disastrous decision that killed tens of millions, for anyone curious. This matches my own understanding.
Of course. The episode you're referring to is one of the most famous and tragic chapters of China's Great Leap Forward (1958-1962). It wasn't just a single moment but a nationwide frenzy of exaggeration, and Mao Zedong's belief in these claims was a critical factor that fueled the disaster.
This phenomenon was centered around the creation of so-called "Sputnik Fields" (卫星田, Wèixīng Tián).
Here is a breakdown of the episode: the context, why it happened, Mao's involvement, and the devastating consequences.
The Context: The Great Leap Forward
In 1958, Mao launched the Great Leap Forward, a campaign to rapidly transform China from an agrarian society into a socialist industrial power. The atmosphere was one of intense revolutionary fervor and political pressure. The core belief was that sheer willpower and the collectivized power of the masses could overcome any material or scientific obstacle.
The slogan of the day was "more, faster, better, cheaper." Officials at every level were under immense pressure to report fantastic successes to prove their revolutionary credentials and avoid being labeled a "right-wing conservative."
The "Sputnik Fields" and the "10,000 Jin" Claim
The term "Sputnik Field" was inspired by the Soviet Union's recent launch of the Sputnik satellite in 1957. The name implied that China was achieving miraculous, space-age breakthroughs in agriculture.
The claims started in early 1958 and escalated rapidly:
- First, a county reported a winter wheat yield of 2,105 jin per mu.
- Soon after, another reported 3,500 jin per mu.
- By the summer, the numbers had become astronomical. A commune in Henan province famously claimed a yield of 7,320 jin per mu.
- This was quickly topped by claims of 10,000 jin per mu (万斤亩, wàn jīn mǔ), and eventually, the official People's Daily newspaper reported a record-breaking rice yield of over 130,000 jin per mu in Macheng County, Hubei.
(For context: 1 mu is about 1/6th of an acre or 667 square meters. 1 jin is half a kilogram or 1.1 lbs. A good, realistic yield at the time was around 400-500 jin per mu. 10,000 jin per mu is equivalent to about 75,000 kg per hectare, a yield that is still physically impossible for rice or wheat today.)
How Were These "Miracles" Faked?
These incredible yields were elaborate hoaxes created for visiting officials and journalists. The most common method was to:
- Select a small, visible plot of land—the "Sputnik Field."
- Just before the visit, cadres would secretly uproot mature crops (like rice or wheat) from many surrounding fields during the night.
- They would then transplant and cram all these crops onto the single small plot, making it appear incredibly dense and fruitful.
The most iconic and notorious propaganda photo from this era shows several children standing on top of a field of wheat, seemingly held up by the dense stalks. In reality, the children were standing on a wooden bench hidden beneath the transplanted wheat.
Why Did Mao Believe It?
Mao's conviction was not born out of simple gullibility but from a combination of factors:
1. Political Atmosphere and Information Bubble: Mao was surrounded by officials who told him what he wanted to hear. Reporting bad news was politically dangerous, so a tsunami of good news, however fake, flowed to the top. When he toured the countryside, he was only shown the carefully staged "Sputnik Fields."
2. Ideological Zeal: Mao deeply believed in the power of socialist collectivization. He thought that by reorganizing society and unleashing the "creative power of the masses," things previously thought impossible were now achievable. The exaggerated claims confirmed his worldview.
3. "Scientific" Justification: The claims were given a veneer of scientific credibility. This came from two main sources:
- Lysenkoism: A discredited Soviet pseudo-agricultural theory promoted by Trofim Lysenko. It rejected genetics and promoted techniques like "close planting" and "deep plowing," arguing that planting seeds extremely close together would make them cooperate rather than compete. Chinese officials claimed their high yields were due to these methods.
- Qian Xuesen's Endorsement: This was perhaps the most crucial element. Qian Xuesen was a brilliant, US-educated scientist and the father of China's missile program. In 1958, he wrote an article published in China Youth Daily where he did a rough calculation based on the amount of solar energy that falls on a mu of land. He concluded that, theoretically, it was possible to convert that energy into a crop yield of over 40,000 jin.
When Mao was told about the fantastic yields, he was initially skeptical. But when he was told that Qian Xuesen, China's most respected scientist, had confirmed it was theoretically possible, his doubts were erased. He reasoned that if it was scientifically possible, then the revolutionary spirit of the Chinese peasants could make it a reality.
The Catastrophic Consequences
This mass delusion had a direct and horrifying outcome: The Great Famine (1959-1962).
The central government, believing these fantastical reports, set impossibly high grain procurement quotas. Local officials, to prove their "success" and meet the quotas, seized virtually all the grain from the peasants, including their food, animal feed, and even the seeds for the next planting season.
The peasants had produced a normal amount of food, but because the state believed they had produced 10 or 20 times that amount, their entire harvest was taken away.
The result was the worst famine in human history. With no food left in the countryside, tens of millions of people starved to death. The episode of the "10,000 jin per mu" was not a harmless propaganda folly; it was a direct cause of this unprecedented tragedy, a testament to the dangers of when political ideology completely detaches from reality.
Some quotes from this:
Qian Xuesen's Endorsement: This was perhaps the most crucial element. Qian Xuesen was a brilliant, US-educated scientist and the father of China's missile program. In 1958, he wrote an article published in China Youth Daily where he did a rough calculation based on the amount of solar energy that falls on a mu of land. He concluded that, theoretically, it was possible to convert that energy into a crop yield of over 40,000 jin.
When Mao was told about the fantastic yields, he was initially skeptical. But when he was told that Qian Xuesen, China's most respected scientist, had confirmed it was theoretically possible, his doubts were erased. He reasoned that if it was scientifically possible, then the revolutionary spirit of the Chinese peasants could make it a reality.
The central government, believing these fantastical reports, set impossibly high grain procurement quotas. Local officials, to prove their "success" and meet the quotas, seized virtually all the grain from the peasants, including their food, animal feed, and even the seeds for the next planting season.
My thoughts are similar to yours although I'm more willing to tolerate posts that you call "almost painfully anti-rational" (while still wishing Said was around to push back hard on them). I think in the early stages of genuine intellectual progress, it may be hard to distinguish real progress from "bullshit". I would say that people (e.g. authors of such posts) are overly confident about their own favorite ideas, rather than that the posts are clearly bullshit and should not have appeared. My sense is that it would be a bad idea to get rid of such overconfidence completely because intellectual progress is a public good and it would be harder to motivate people to work on some approach if they weren't irrationally optimistic about it, but equally bad or worse if there was little harsh or sustained criticism to make clear that at least some people think there are serious problems with their ideas.
- One of the worst examples of this "psychosis", in terms of consequences, is the 1959-61 Great Chinese Famine which killed between 15 and 55 million people.
- We should heavily discount negative feelings related to criticism, instead of taking them at face value (as showing that something is wrong and should be fixed, e.g. by getting rid of the source of the criticism). I think this can often manifest not as "I hate this criticism" but more like "This person is so annoying and lack basic social skills."
- There's probably an effect where the less criticism we hear, the more sensitive we become to the remaining criticism, suggesting a slippery slope towards being surrounded by yes-men.
- Remember that most CEOs had to work their way up to that position, and have seen sycophancy from the bottom and understand that it's bad, but still fall prey to this problem.
Most organizational leaders have people they have to answer to and could be replaced if they perform badly. Why doesn't this fix the problem? Trying to answer this myself, I found this paper which says:
In a series of studies, Westphal and Stern (2006, 2007; Stern and Westphal, 2010) showed that ingratiation by a manager or director toward the chief executive officer (CEO) of a large company has a strong, positive effect on the likelihood of receiving the CEO's recommendation for a board appointment at another firm where the CEO serves as director.
I guess this is an instance of a more general phenomenon known as "board capture", which helps explain why having a board doesn't solve the whole problem. The paper also says:
Further analyses suggest that strategic persistence that results from high levels of flattery and opinion conformity directed at the CEO can result in the persistence of low firm performance and may ultimately increase the likelihood of the CEO's dismissal.
So I guess the CEO being fireable does solve the problem to some extent.
1
I think this 2023 comment is the earliest instance of me talking about turning down investing in Anthropic due to x-risk. If you're wondering why I didn't talk about it even earlier, it's because I formed my impression of Dario Amodei's safety views from a private Google Doc of his (The Big Blob of Compute, which he has subsequently talked about in various public interviews), and it seemed like bad etiquette to then discuss those views in public. By 2023 I felt like it was ok to talk about since the document had become a historical curiosity and there was plenty of public info available about Anthropic's safety views from other sources. But IIRC, "The Big Blob of Compute" was one of the main triggers for me writing Why is so much discussion happening in private Google Docs? in 2019.