LESSWRONG
LW

Comment Permalink
RHollerith110

China has put an extraordinary amount of resources into developing their own chip fabrication capacities, and so far has failed pretty pathetically and now IMO it is probably too late: China will more likely than not never acquire the capability to design and fab GPUs and TPUs as efficient as those currently being designed by Nvidia and Google respectively and currently being fabbed by TSMC and whoever is fabbing Google's TPUs.

The ability to design and fab such chips is currently a "world ability": the fabbing relies on litho machines made by in the Netherlands, which in turn depends heavily on lenses made by Zeiss in Germany and other advanced products from Japan, the US and probably other countries. The fabbing relies on very expensive other products that like the litho machines are made by just a handful of companies, almost all of which are vulnerable to pressure from the US government (especially if other governments generally aligned with the US tend to agree with the USGov's reasons). Designing the chips (the layouts) requires software available from only a handful of suppliers again probably all vulnerable to pressure from the USGov. It is currently a world ability and more likely than not will remain a world ability and not an ability of any one bloc (e.g., the Chinese government and its friends) unless we consider the US and Europe and entities vulnerable to pressure from those 2 as one bloc.

Note the "more likely than not" above! I'm not saying that I can reliably predict the relative strength of China in the future! It's just that I am not moved by the numerous pronouncements that the rise of China is inevitable. To take just one example, Peter Zeihan says that if China were subjected to the same sanctions imposed on Russia after Feb 24 of this year, there would be rolling blackouts within a few months, trucks running out of gas and abandoned on its highways and in about a year half of the population will have died of starvation: in contrast to Russia, which produces more fossil fuels, fertilizer and basic foodstuffs like wheat than it needs, China is heavily dependent on imports for those things.

Zeihan goes on to say that although China has always been aware of the possibility that US and Europe governments would impose sanctions on it, it was utterly shocked to learn of the possibility that major corporations would in the future refuse to do business with China without even being forced to by a government. He says he has heard from sources inside China that such an eventuality was never considered by Chinese planners and now they feel they must re-evaluate their entire geopolitical strategy.

There is a danger when spending a lot of time reading comments on the internet -- or when reading the output of our newspapers and similar institutions -- to allow the sheer repetition of a claim thousands of times from thousands of individual authors to give the claim more credence than it deserves. The vast majority of the claims I have seen as to the probable eclipsing of the US by China either makes no argument in support of the claim or the included argument is very unimpressive. Yes, there are experts I respect, like John Mearsheimer, who make the claim (though I haven't heard him make an argument for it: he just claims it, but he is Mearsheimer, so I listen), but there are other geopolitical experts I respect, like George Friedman, who believe that China will be lucky if 15 years from now it hasn't split into 2 or 3 countries -- and that the Chinese government knows that and preventing it is their main focus. Friedman also says that the Chinese navy is still not capable enough to protect the ships that bring it oil from the Persian Gulf from pirates and privateers if the US navy stepped down from its role of protecting the world's shipping from pirates. "Privateers": if there ever is a really serious oil shortage, the Indian government might follow a centuries-old tradition of defining the takeover of oil tankers bound to China as "piracy", but rather "privateering", and consequently not illegal and in fact maybe it is rewarded, and according to Friedman, there is nothing the Chinese navy can do about that: they simply lack the ability to project force that far from port. Friedman also says that the Japanese navy would more likely than not prevail over the Chinese navy if they fought today. He says that that is partly because Japan's admirals were trained by admirals who were trained by admirals who actually fought a major war with what remains the dominant naval technology relevant to a major war, namely, the aircraft carrier -- and according to him, that that counts for a lot.

The reason I'm commenting is that this has relevance to managing the risk of AI: in particular, in a situation as dire as the current global situation around AI research, we should be paying attention to those futures in which our ability to control and manage the global situation around AI has drastically increased even if right now those futures seem unlikely. In particular, there is a chance that the leaders of China will in the future do something at least as risky and ill-fated as Putin's decision to invade Ukraine, and the response to that by the West will make it so that we don't have to worry about AI researchers in China because China will be too preoccupied with feeding itself and just keeping the country supplied with electricity and whatever China uses to heat its homes. Or maybe Southern China or Shanghai will make a serious attempt to secede from Beijing's rule, in response to which Beijing will lock the country down kinda like they did for Covid, with the same effects on the Chinese economy that the Covid lock-downs had, but these lock-downs will go on for years, and again then we don't have to worry about the danger posed by Chinese AI researchers.

So if young people concerned about AI risk move to Washington and London and Paris and start a career in the national-security bureaucracy there with the goal of eventually convincing their government to impose a regime of "compute governance" similar to the existing regime of nuclear non-proliferation, then maybe their efforts will be in vain because the center of mass of irresponsible AI research (currently somewhere in the Atlantic between the US and Britain) will just move to China. But maybe their efforts won't be in vain because China will stop being able to host the intensity of scientific research and technological development necessary to contribute significantly to the AI danger. And maybe Russia will continue its decline with the result that regardless of the wishes and the ambitions of its government, it becomes unable to contribute significantly to the AI danger (which of course its government does not perceive as particularly dangerous).

Reply
See in context
by aog
5 min read
58

102

What are the restrictions?

Broadly speaking, the policy restricts the sale of GPUs and related technology to Chinese companies. The specifics remain unclear because (a) the Department of Commerce hasn't released its official press report yet and (b) licenses to sell compute to Chinese companies will be approved on a case-by-case basis. 

Here's the NYTimes description of the restrictions: 

Companies will no longer be allowed to supply advanced computing chips, chip-making equipment and other products to China unless they receive a special license. Most of those licenses will be denied, though certain shipments to facilities operated by U.S. companies or allied countries will be evaluated on a case-by-case basis, a senior administration official said in a briefing Thursday.

The restrictions limit U.S. exports of the cutting-edge chips called graphic processing units that are used to power artificial intelligence applications, and place broad limits on chips destined for supercomputers in China. The rules also ban U.S.-based companies that make the equipment used to manufacture advanced logic and memory chips from selling that machinery to China without a license.

Perhaps most significantly, the Biden administration also imposed broad international restrictions that will prohibit companies anywhere in the world from selling chips used in artificial intelligence and supercomputing in China, if they are made with U.S. technology, software or machinery. The restrictions used what is know as the foreign direct product rule, which was last utilized by former President Donald J. Trump to cripple Huawei.

Another foreign direct product rule bans a broader range of products made outside the United States with American technology from being sent to 28 Chinese companies that have been placed on an “entity list” over national security concerns.

Those companies include Beijing Sensetime Technology Development Co, a unit of major Chinese artificial intelligence company, SenseTime. Also included are Dahua Technology, Higon, IFLYTEK, Megvii Technology, Sugon, Tianjian Phytium Information Technology, Sunway Microelectronics and Yitu Technologies, as well as a variety of labs and research institutions linked to universities and the Chinese government.

The rules also restrict U.S. citizens from helping to develop the Chinese semiconductor industry to advanced levels. Earlier on Friday, the administration announced that it was adding another 31 Chinese companies and institutions to an “unverified list” that limits their ability to obtain a smaller set of certain regulated U.S. items. Among them is Yangtze Memory Technologies Co., Ltd, a major memory chip maker from which Apple has considered sourcing some products.

In a briefing with reporters, senior administration officials said the measures would be limited to the most advanced chips, and thus would not have a broad commercial impact on private Chinese businesses. But they conceded that they could become more restrictive over time, given that technology will begin to outpace them.

Licenses to continue sales to China will be approved on a case-by-case basis by the Department of Commerce. Standards for these licenses will therefore be an ongoing battleground where this policy can become more or less strict. NYTimes:

Some Republican lawmakers and China hawks have criticized the department for being too willing to issue such licenses, allowing U.S. companies to continue selling sensitive technology to China even when national security may be at stake.

“If you want to stop it, you can just stop it,” said Derek Scissors, a senior fellow at the American Enterprise Institute. “When you create a licensing requirement, you are announcing to the world we don’t want to stop it. We are just pretending.”

The waivers to continue selling products to China could come under more scrutiny if Republicans regain the majority in the House after midterm elections.

Representative Michael McCaul, Republican of Texas, said he intended to use his authority as current ranking member of the House Foreign Affairs Committee to press the Bureau of Industry and Security, which reviews such licenses, on the applications and decisions.

Policy Environment

The US has taken complementary policy actions recently, including banning NVIDIA from selling A100s and H100s to Chinese companies and blacklisting 13 more Chinese companies from receiving any investment by Americans. 

Taiwan seems to be on-board with the plan, promising "very firm" export controls on Taiwanese chips being sold to the Chinese military complex. Taiwan is a crucial player in this battle, home to Taiwan Semiconductor Manufacturing which produces 53% of the world's outsourced chips and all of NVIDIA's GPUs. 

The most important background is the CHIPS Act passed this August providing $52B for US chip manufacturing. Here's a Wikipedia summary of the spending: 

The CHIPS Act includes $39 billion in tax benefits and other incentives to encourage American companies to build new chip manufacturing plants in the U.S.[7] Additionally, $11 billion would go toward advanced semiconductor research and development, with $8.5 billion going to the National Institute for Standards and Technology, $500 million for Manufacturing USA, and $2 billion for a new public research hub called the National Semiconductor Technology Center. $24 billion would go to a new 25 percent advanced semiconductor manufacturing tax credit to encourage firms to stay in the United States, and $200 million would go to the National Science Foundation to resolve short-term labor supply issues.

How will China respond?

Unclear. NYTimes is maximally vague: 

It remains to be seen whether the Chinese government will take action in response. Samm Sacks, a senior fellow at Yale Law School who studies technology policy in China, said the new rules could push Beijing to impose restrictions on American companies or firms from other countries that comply with U.S. rules but still want to maintain operations in China.

“The question is: Would this new package cross a red line to trigger a response that we haven’t seen before?” she said. “A lot of people are anticipating it will. I think we’ll have to wait and see.”

While the US is dependent on China for many imports, we do not seem to critically depend on their GPUs. (I don't have strong proof of this claim and would welcome disagreement. My understanding is that most US GPUs are designed in the US and manufactured in Asia outside China using machinery developed in the Netherlands and elsewhere around the world. See here.) 

China might respond in turn with similar GPU export controls against the US, but given the limited reach of Chinese compute in the US, I expect the US would welcome those restrictions as they further prevent American dependence on Chinese supply chains. A stronger response could include export controls against the US in other sectors or other actions against US foreign policy interests around the world. 

Over the coming years and decades, China could gain more influence over Taiwan, South Korea, Singapore, Japan, and other Asian countries with strong chip manufacturing. This could be a real problem for the US which depends on exports from those countries. But promoting US chip independence by funding domestic manufacturing and cutting ourselves off from global supply chains seems like a good way to counter that threat. 

How does this support US strategic goals?

These policies together support independence for the US chip supply chain. This is an important long-term goal for the US given that much of the world's compute is produced in East Asia where China could attempt to restrict our access. It's part of a larger strategy of economic decoupling from China in preparation for another Cold War. (Another policy that would promote US supply chain independence is preventing US companies from using Chinese parts in building chips -- perhaps this is on the agenda, or will be enforced unilaterally by the Chinese government in retaliation to our policies.)

The policies also slow down Chinese AI. Some of the companies that are specifically targeted seem particularly heinous such as SenseTime and Dahua which provide facial recognition software for the concentration camps in Xinjiang, while others are more neutral, such as AMD's partnership with Chinese companies to build CPUs. From a US national security perspective, slowing down Chinese growth in general and military AI growth in particular seem widely accepted as a good strategy. I would expect the same thing from an x-risk perspective, though perhaps decreasing Chinese reliance on US chips only reduces our influence during a future critical time (see Matt Yglesias). 

AI GovernanceAI
Frontpage

102

Mentioned in
Analysis: US restricts GPU sales to China
New Comment
58 comments, sorted by
top scoring
Click to highlight new comments since:
[-]gwern*1163

It looks like this was only the start. The news from the past few days is alarming at face value: it sounds like a near total ban on everything, TSMC fabbing only the start (!), now all ASML customer support† is gone (!!) not just EUV machines, on top of which all the American-citizen employees (a nontrivial fraction due to education/birth abroad) have halted work literally overnight (!!!), and overall what sounds like the collapse of Chinese semiconductors amidst a Chinese economy already showing serious signs of distress and little capacity to keep an industry on life-support indefinitely as they scramble to survive - and once they go down in a cascade of bankruptcies/liquidations*, each node killing the nodes dependent on it, and everything is sold off and liquidated and employees scatter to the four winds, rebuilding that semiconductor ecosystem (which has already cost $100b+ and decades to build in repeated efforts) years later, when even further behind, will be more like starting from scratch than turning on an idle car. (And how long might that be, if China is entering its 'lost decades'...?) What's the exit plan here for the US, what does success look like? If there is one thing to learn from past attempts at strategic embargoes like China's failed rare earth embargo or Russia's failed gas extortion, it's that while they can be extremely effective in the short term, they are one-shot weapons which neutralize themselves in the less short run.

Or to put it another way, if the CCP tries to invade Taiwan in the next 10 years, historians are likely going to point to 2-3 days ago as the pivot: it is now a razor blade cutting China's throat if AI and high tech in general is the future, with nothing more to lose and everything to gain from destroying TSMC so no one else can have it either. (Meanwhile, the NYT front page story: "Companies Are Hoarding Workers. That Could Be Good News for the Economy." Glad our media is still doing a bangup job of keeping everything in perspective.)

The implications for AI scaling and timelines are also immense: aside from the obvious disaster for Chinese AI research, which will stagnate at present Chinchilla-level runs, any invasion attempt is tantamount to the destruction of TSMC (you can now be sure some cruise missiles will find their way to TSMC chip fabs conveniently set out as hostages on the western side of Taiwan & most definitely not hardened underground in Taiwan's mountains) and will set back scaling by years, possibly decades given the realities of chip fab, capital investment, globalized supply chains, tacit knowledge, experience curves, and risk premia leading to compounding falling behind exponential projections.

So uh... yikes? Is this really what's happening? Is there some ameliorating factor not covered in any of the reporting so far? We should all probably be paying way more attention to this than Elon Musk's latest bipolar antics, or even Putin's nuclear blackmail, because this appears to actually be happening right now, not merely possibly some day. We appear to live in interesting times.

Links (in no particular order):

(BGI Genomics also got blacklisted, not that anyone particularly cares about Chinese genetics at this point.)

* One might think it'd be crazy to try to trigger this sort of systemic crisis in the middle of a global, and Chinese, economic crisis. But of course, like chemotherapy, the question isn't whether it's bad for you, but worse for the other guy, and potentially pushing the chip ecosystem into a systemic collapse will never be easier than it is today. The USA is much wealthier than China, it can handle high-priced chips better.
† Presumably this extends to updates, upgrades, replacement of consumables, repairs when things break, replacement of broken machines... (How's Russian military manufacturing & aviation going these days?)
‡ What an utter insult to Xi Jingping, incidentally. He must be furious to have this drop literally days before. Brother Pooh is not noted for his thick skin. I wouldn't be surprised if whoever is masterminding this in the Biden administration timed it deliberately - after all, given that it's been several weeks since the TSMC GPU embargo was announced, they could easily have delayed it to this week & after.

Reply
[-]gwern*1147

So, it's been about a month and a week. Seems like enough time to evaluate a little the link dump above (which is in semi-chronological order). Where is this chip embargo now? I'd summarize the news & expert opinion thus far as:

  1. The embargo is still solid.

    The Biden administration has not walked it back, and confirmed the more restrictive parts. I have not seen any coverage indicating that Chinese corps are trivially circumventing it, both ASML & TSMC seem to be enforcing it, and Chinese corps are biting the bullet in deliberately gimping their chip designs to comply with it. Further, major players like Apple have been canceling equally major orders. None of this would be happening if it were only on paper or could be easily circumvented with a shell corp or something.

  2. The consequences for the Chinese chip industry are still big, and bad.

    We have plenty of reports about major layoffs, large hits to revenue, cutbacks to investments/R&D, exodus of US/Taiwan-linked personnel, and a halt to VC investment. Quotes from insiders like VCs or major chip manufacturer representatives range from 'dire' to 'apocalyptic', with time-ranges in the years for when---hopefully!---things might be better again. (Much less informatively: My previous comment got some circulation on social media, and mockery aside, there weren't any comments I saw that looked like good arguments for why the impact would be minimal, rather than vague assertions that they would just somehow be fine even if they couldn't get any ASML gear etc. I also asked anyone who might know something on my recent SF trip why this might not be a big deal, and got nothing, and overall an impression that everyone has been too distracted by the numerous other things happening of late to really process it.)

    How big? It's unclear because these aren't great times for the global semiconductor industry either, as they are running into the general economic malaise and the bullwhip effect where the excess COVID-induced demand & past supply shortages are fading out---but it's worth noting that it seems like non-CCP firms like Nvidia & TSMC were well-aware that they were going to overshoot to some degree and prepared for it, and don't seem to be in nearly as bad shape overall.

  3. But there does not (!) seem to be any massive CCP bailout of the Chinese chip industry planned.

    While there are many fiscal stimuli ongoing, including ones announced since the embargo, chip-specific ones have not been announced---as would be necessary both to coordinate and restore confidence in the ecosystem---and the reported layoffs/cutbacks are highly costly mistakes if you expect a big faucet of billions of dollars of free government money to be turned on any month now, so seem to imply that the post-embargo meetings with the government did not spur a bailout effort. If Bloomberg's reporting is correct (and I have no problem believing that a meeting attended by that many figures had at least one person willing to recount it all near-verbatim to a Bloomberg journalist), then they already collectively told the CCP that they were 'doomed' without massive additional investment, and the CCP appears to've shrugged and told them that 'domestic IT market demand was adequate' for them to survive.

  4. The prospect of a collapse, beyond merely a hard recession, remains unclear, and will be hard to evaluate.

    It may be tempting to say that "well, it's been a month and while they've had some painful blows they are clearly still fine". But that's never how systemic collapses happen---remember bubbles like the Japan bubble, the US housing bubble, fracking, the repeatedly-averted Chinese bubble popping etc, or consider cryptocurrencies right now: the industry seemed to have weathered its most recent bubble popping with surprisingly few casualties compared to the historical fallout of each cryptocurrency bubble, in part due to bailout purchases/investments by sterling household brands like FTX... and then literally overnight that fell apart, and many individuals at many entities received unwelcome surprises about what connections there were in the cryptocurrency ecosystem. "There is a great deal of ruin in a nation."

    Almost all entities involved still have runway: I mean, if you were so fragile that you had less than 1 month of expenses (in the worst case of abruptly going to zero cashflow) and could have gone bankrupt already, then you were already doomed, embargo or no embargo. It is just very little time, on an industry-wide scale. Zombie companies can stagger on for a long time before finally going bankrupt. (As the quote goes: "slowly, then suddenly.") Cash has not run out, reality has not set in, optimism remains high, supply stockpiles are only partially depleted, complex machines have not yet broken down or reached the end of maintenance cycles or expected lifetimes, slashed orders mean capacity losses are less important... Similarly, when Putin invaded Ukraine 275 days ago, they were extensively embargoed, particularly on chips, and there have not been any dramatic consequences with screaming headlines about passenger planes falling out of the sky---but there have been consequences, such as the gradual disappearance of all their good high precision missiles/artillery like cruise missiles, the choking off supplies to their front lines, resorting to cannibalizing many units to get one working unit or using very expensive equipment in incredibly wasteful ways (hypersonic missiles on apartment blocks, anti-ship missiles on land targets, S-300 AAs as ghetto cruise missiles), obsolete equipment & reliance on imports like Iranian drones, and factories you've never heard of simply dying for lack of chips & other supplies because they were unable to work around so many dependencies lopped off all at once. It has taken many months for subtle signs to show up of real damage, and indeed, many people early on were quite skeptical any Russian embargoes could do any good (surely they will just import it from China---!).

Points #1/2/4 are no surprise but point #3 is a surprise. I think pretty much everyone took for granted that the basic CCP response would be to double down on all the chip subsidies, and if they'd already dumped in $100b, oh well, now they'd dump in $200b (in for a penny, in for a pound). This seems... not to have happened? That's surprising. At least, I'm surprised. If you aren't surprised, why not? Is there a bailout somewhere I missed in the news?

Reply
[-]gwern*56-5

So, let's take as a hypothesis that there is no bailout for chips, especially on a large enough scale to really matter. Why this neglect?

Let's take further as a hypothesis that the reason for the neglect is not simply Xi deciding to invade Taiwan & therefore writing off the domestic chip industry, as this is an extreme course of action and one that most people claim to find even more improbable than, say, the idea that a Xi-like dictator like Putin would do something as absurdly self-destructive as actually invade Ukraine this year instead of just saber-rattling his massed troops along the border to intimidate them into concessions.

The remaining conclusion would seem to be that Xi has chosen to take the L: he is neither going to massively bail out the domestic chip industry nor take out their competitor, and is just going to let it take its lumps and whatever happens happens, China will just have to get along with whatever chips it can make and the gimped chips TSMC will deign to manufacture for it. (At least, I can't think of any additional meaningful choice outside the trilemma of 'bailout, accept defeat, or invasion'.) OK, but why? I struggled to see how that could make sense. Let's go even further into the hinterlands of geopolitical & psychological amateur speculation and ask what Xi is thinking...

From the scaling-pilled perspective, or even just centrist AI perspective, this is an insane position: it is taking a L on one of, if not the most, important future technological capabilities, which in the long run may win or lose wars. If China wants to dominate Asia, much less surpass the obsolete American empire, or create AGI, or lead in aerospace, or create '5G' or whatever, it's hard to see how it's going to do that while paying more for chips which are half a decade or worse out of date.

But Xi is not scaling-pilled (after all, few people are, even in the most cutting-edge AI R&D labs). So maybe we should ask: is he centrist on AI? Er... Oh - does he care about AI at all? What evidence is there that he does? There doesn't seem to be much. Going further: what evidence is there that he even regards chips in toto as being all that important? From Xi's perspective, what has 'chipolitics' looked like thus far?

  1. Huawei: the US embargo of chips to the national champion Huawei, and Huawei's near-death experience, is where chiplomacy started getting ugly. This involved ordinary bulk consumer chips for smartphones and equipment like 5G base stations. It did not involve high-end GPUs or future chips at all. Huawei simply needed millions of chips to sell for export of hundreds of millions of smartphones to the entire world like Africa, and couldn't get them. But, they survived, and they may now struggle to get the chips that they would like and rely on alternate suppliers of lower-end chips (any dreams of challenging Apple on its high-end home turf are long gone), but Huawei as a whole does still sell a ton.

  2. Russia: another embargo post-Ukraine, cutting off supply of all sorts of chips, almost all antiquated chips designed decades ago for specialized equipment: again, nothing like a H100 GPU or in any way connected to stuff like '4nm nodes'. Even for drones, you can usually get by with pretty old parts or improvising; it's more important to have lots of cheap drones than geewhiz. Here too the problem is Russia needs (1) millions upon millions of specific chips to feed into existing manufacturing lines to feed the meatgrinder of its 'eastern front' (if you will), (2) needs them very soon, preferably a few months ago, as the Ukrainian assault will not stop and they have burned through much of their reserves, and (3) cannot get millions of those specific chips whether a few months ago or a few months from now.

  3. The current US/TSMC chip ban: little of the damage reported thus far has much to do with failing to develop new nodes or not getting access to A100/H100s. Nvidia canceled a lot of orders of them, but I haven't seen anything report about big corporations going out of business etc, and Chinese AI research seems more or less to be carrying on as before with its current stock. The damage is coming from losing big bulk orders like dumping memory to Apple and from catchup designs not being fabbed and suppliers of existing stuff being knocked out. As time passes, these 'seen' damages will be replaced by much larger 'unseen' damages cascading out.

    Even from an AI research perspective, the damage will be subtle. You can keep on using existing clusters indefinitely; they will just put a ceiling on what you can do with reasonably trained or sized models before the communications delays kill performance, and they will hamstring your budget by costing several times more over the next few years, increasingly so. You can assemble clusters 1 GPU at a time, buying them on the gray market, eventually, years later, reaching a respectable size. You can keep doing AI research which looks like almost all AI research in 2021*, and which will look respectable, and use hand-me-down public models from the West, and do lots of small-scale optimization work (which will be, or has already been, Bitter-Lessoned but that is cold comfort to those who can't afford the bigger irons). You can keep pumping out CS conference proceedings with low-grade small-budget research that would barely make a decent blog post. (I read a lot of IEEE papers relating to anime research, typically GANs, typically terrible, typically East Asian, and typically a few GPUs max.)

    A chip ban doesn't doom Chinese DL (or Chinese AI in general) research to becoming a cargo cult field, but it does render it increasingly prone to irrelevancy, brain drain of everyone who wants to work on the future of DL and not the past, involution / l'art pour l'art, and organizational/intellectual sunk costs and Galapagos syndrome. Considering that Chinese R&D and science are not particularly healthy or fraud-free in the best of times... But how would an outsider, such as a political leader, notice that the cargo is no longer landing?

* I realize people like to portray AI scaling as some overwhelmingly dominant paradigm. This isn't true. You can go look at a page of NIPS abstracts and see that that is not the case, or look at the NLP survey the other month and note how few people will endorse scaling propositions on a mere anonymous survey, or note how few Arxiv submissions a day merit a /r/MLscaling submission despite spreading a broad net. AI scaling is far from the majority of AI research; scaling research is merely the majority of research that will matter.

We could add a few other points:

  • the senior CCP leadership is semi-famous for being 'technical' (typically engineering degrees like hydrology or mechanics or aerospace) but little to do with anything computer. Xi Jinping has a degree in chemical engineering from a low-rigor period 43 years ago, and then a degree in BS, both of which might just be mostly fake (pretty common). Propaganda aside, his major intellectual interest is literature, particularly Goethe. He has not overseen any major technical projects, or made any major intellectual contributions I'm aware of; indeed, reading about him, he's always struck me as being mediocre in every aspect besides Comunist Party infighting, bureaucracy, and consolidating power.

  • Techlash: Xi's reign has been marked by constant hostility towards and suppression of 'big tech' ie. the very people who would benefit and use cutting-edge chips the most and who would be explaining to Party officials the long-term prospects. There are innumerable angles here (for example, yesterday, it sounds like the CCP may deign to graciously allow a few video game developers to, after a year or two of it simply not being allowed, 'release video games'), and suppression of individuals like Jack Ma come off as very personal. The rhetoric of the regime emphasizes redistribution, only thinly veiled as 'voluntary donations', and the Party defending the public and the 'China Dream' from rapacious corporations.

  • Conversely, his reign has been marked by an emphasis on legible atom-heavy scientific projects, and a general downplaying of everything related to bits or information, unless it has a national security angle (leading to 'Dutch disease' where an ultra-niche like facial recognition gets lavishly funded, crowding out more generalizable research). For all the talk of 'data is the new oil' or 'China as a data superpower' or the advantages from 'Chinese lack of privacy', China still drastically underperforms in making good use of it. There are large GPU clusters; but all the really important DL research still comes from the West - I've noted that it seems like you could trade all of Chinese DL research impact for 1 or 2 Western labs like DeepMind's impact and still have enough change left over for coffee. There are large numbers of genetic datasets fragmented over many Chinese groups; but global genetics research remains driven by UK Biobank from 2014 etc.

    • Attempts at scientific reform and quality control are on uneven ground. Fraud in Chinese science seems to still be endemic, and impact low; past efforts like paying per publication merely made the problem worse. In ML, when fraudulent rings of peer review are discovered, they often involve Chinese conferences or reviewers. Publication in prestigious foreign journals like Science or Nature (rather than Chinese journals) remains a major goal, as a matter of quality control, because domestic Chinese journals cannot be trusted.

  • There seems to be considerable contempt for the USA and American capabilities in China among 'wolf warriors', taking cues from the top, and with considerable historical precendent for authoritarian countries to mistakenly gauge the USA as 'decadent' and 'weak'. This may have been trimmed a bit after Ukraine and seeing what things like HIMARSs can do, but it runs deep and inside the Chinese bubble, there is little correction. (When was the last time Xi Jinping was in the USA and saw more than political flunkies? Or any Chinese, for that matter, given their multi-year near-shutdown of international travel?) The thinness of the air at the heights Emperor Xi inhabits is prone to induce altitude sickness and hallucinations. (But at least, thanks to "Zero COVID", among his many problems, personally getting COVID is not one of them.) If the Americans are decadent because of their emphasis on software and compute, and China & Xi are superior because they aren't decadent...

  • The many stimuli the CCP has used repeatedly over the past decade, and since the chip embargo was put in place, shows that they can and will and have had enough time to do so, yes, but the flip side is that the more you stimulate, the less ammunition you have left for the next stimulus, and the less credibility you have to the populace or markets. At some point your macroeconomics start looking bad. There may not be as much money left as one would assume.

So, in past chip incidents, the primary problem has been a complete absence of any chips, and not so much the advancement of the chips themselves. He has never seen anyone lose a war due to lack of AI or GPUs; he's only seen disasters caused by lacking perfectly ordinary chips that his domestic manufacturers probably could've made 10 years ago. And in learning lessons from past chip incidents, what Xi brings to the table is: zero technical competence or expertise in the relevant area, a hatred of software and everything to do with it, and long-standing prioritization of heavy-industry-like stuff (which is clearly visible to the naked eye and 'conventional' and 'prestigious' and applauded by old credentialed foreigners).

Further, mistakes in this regard may be hard to see. 'The seen and the unseen' is a dangerous trap because it is so much easier to see the seen than it is to see the unseen. If Xi makes a mistake on chips, a military mistake, then by the nature of things military, he may never realize it. If the engineers of, say, hypersonic missiles can't get enough high-end GPUs, their complaints will be ignored by the next layer of management and never punted all the way up to Beijing, and they will simply run their simulations at a lower resolution or take other shortcuts; and if the hypersonic missile in question turns out to be a lemon, inadequate to hit NATO units or US aircraft carriers, how will anyone ever find out short of a war over Taiwan---at which point it is far too late? Naturally, of course, given a supply of at least basic chips to work with, the establishment will assure him everything is fine, just like the Russian military assured Putin it was not a paper tiger or hopelessly undermined by corruption, and almost all the time they will be right.

That is, from Xi's perspective, all in all, maybe it looks fairly reasonable to neglect chips right now. They aren't that important, and don't seem in that much worse trouble than anything else, while bailing them out to the degree where they can potentially gain, or at least near, the cutting-edge would use up a ton of an increasingly skint government's money. Plus, as master of the currents of history piloting China to a glorious Chinese Century avenging the Century of Humiliations, he has much bigger fish to fry, like the house of cards which is real estate, and Zero COVID. There will be side effects, yes, but if gaming GPUs becoming expensive helps turn little Aiguo away from a career as a useless game programmer into a respectable hardworking fusion physicist, perhaps that's even a feature rather than a bug?

Well, I could be wrong about all this. But now I can see at least one perspective from which the chip embargo is a big deal but also Xi's rational response is to indeed just take it on the chin, and perhaps tone down the rhetoric and engage in a bit more biding one's time & hiding one's strength. (I doubt that the long-term aims have changed meaningfully just because Beijing is calibrating its rhetoric a little down from recent peaks of aggression, but in the short term, things will be superficially more peaceful.)

Reply
[-]Rob Bensinger123

From the scaling-pilled perspective, or even just centrist AI perspective, this is an insane position: it is taking a L on one of, if not the most, important future technological capabilities, which in the long run may win or lose wars. If China wants to dominate Asia, much less surpass the obsolete American empire, or create AGI, or lead in aerospace, or create '5G' or whatever, it's hard to see how it's going to do that while paying more for chips which are half a decade or worse out of date.

The scaling-pilled AI view ought to be that scaling AI kills you. Why pretend that there's a strategic advantage here, as opposed to a loaded gun you can point at your own head if you're stupid enough?

It's one thing to say "given China's actual beliefs, they ought to do X" or "if China were rationally acting on a correct understanding of the world, they would do Y". But why criticize China for avoiding a self-destructive action that would make sense to do if they had a specific combination of definitely-true, maybe-true, and definitely-false beliefs -- a specific combination they don't in fact have?

Reply
[-]Kaj_Sotala*68

Isn't the "scaling AI kills you" view the conjunction of "scaling-pilled" and "alignment is extremely difficult" views, rather than being identical with the scaling-pilled view?

One could reason as something like:

  • If alignment is as hard as people make it out to be, we're in all likelihood dead anyway since Westerners are going to develop AI even if we don't.
  • If alignment isn't as hard as people make it out to be, then the country that controls the most powerful AI will be the one that becomes dominant in the world.
  • Thus if alignment is hard it doesn't matter what we do, and if alignment is less hard we should invest in AI. Thus, we should invest in AI.

(There's some obvious nuance that this argument is missing, e.g. the chance of arms races increasing the difficulty of alignment, but some form of it still seems reasonable to me.)

Reply
[-]Wei Dai50

From the scaling-pilled perspective, or even just centrist AI perspective, this is an insane position: it is taking a L on one of, if not the most, important future technological capabilities, which in the long run may win or lose wars.

Are you suggesting that the sane policy is for Xi to dump in as much subsidies as needed until China catches up in semiconductors with the US and its allies? I haven't seen anyone else argue this, and it seems implausible to me, given that the latter collectively has much greater financial and scientific/engineering resources. China's GDP is only $18T vs $58T for OECD, and as you say there's a lot more corruption/fraud in Chinese R&D, so how can they hope to win an outright tech race (and starting from about a decade behind)?

Reply
[-]gwern141

Are you suggesting that the sane policy is for Xi to dump in as much subsidies as needed until China catches up in semiconductors with the US and its allies? I haven't seen anyone else argue this

Yes. And perhaps no one else does because they aren't scaling proponents. But from a scaling perspective, accepting a permanent straitjacket around GPUs & a tightened noose is tantamount to admitting defeat & abandoning the future to other countries; it'd be like expelling all your Jewish scientists in 1935 & banning the mining of uranium. It's not the beginning of any story that ends in victory & being #1, only of stories that end with you being #10, or #100. Like the Manhattan Project, you pay whatever it costs, and it costs what it costs.

so how can they hope to win an outright tech race (and starting from about a decade behind)?

Well, it's certainly not easy. It's not a great situation for China to be in, and yet, it is in fact the one they are in, and they have to deal with it. Reality has no obligation to make anything easy for you, nor should you expect something like 'dethrone the global hyperpower and create a new world order around the Middle Kingdom' to be easy. Similarly, it's not easy to defeat a Russian or Chinese invasion, nor was it easy to develop atomic bombs, etc, but if you don't, there will be consequences you may find unacceptable - even, existential, one might say - so you don't get much of a choice. You miss 100% of the shots you don't take.

China has a lot of resources yet untapped, both financial and otherwise, and is doing many other things like stimulating other sectors of the economy, so clearly it can try more than it is right now - but it appears not to be. If it's not a capability issue, then it must be a choice. Xi seems to disagree that the consequences of choosing to accept defeat in chips & AI will be all that unacceptable, and my comment here is about thinking through what the strategic logic could be from his perspective which makes that choice an acceptable one because the the chip race game is not worth the candle.

Reply
[-]Wei Dai00

I agree with Rob Bensinger's response here, plus it's just a really weird use of "insane", like saying that Japan would have been insane not to attack Pearl Harbor after the US imposed an oil embargo on them, because "You miss 100% of the shots you don’t take." Thinking that way only makes sense if becoming a world or regional hegemon was your one and only goal, but how did that become the standard for sanity of world leaders around here?

Reply
[-]gwern90

It's not a weird use. It is a completely normal one: becoming the hegemon is the avowed goal and so making choices which drop the odds of success so drastically raise questions about the thought process, which in this case I find much better explained by simply mistaken beliefs and desires on Xi's part combined with ordinary sanity. I have already explained at length why I think those beliefs are mistaken, and I also think the desires are bad: do I think China (in the sense of its population as a whole) is better off for Xi's powerhunger, or that China would be better off for launching a scaling Manhattan Project? Certainly not: in my opinion, Xi's reign has made the Chinese people substantially worse off than they would have been under a more status quo leader, and Xi has probably single-handedly curtailed their long-term growth prospects & condemned them to permanent middle income status as they begin to enter a Japan-style malaise, in addition to authoritarian disasters like the Uighurs. But many Chinese believe otherwise now, and endorse further questing for hegemony, and they & Xi are in charge, not you, and desire what they desire, not what you desire - you (and Rob) are projecting your own preferences and moralizing here, instead of trying to understand what is going on.

Reply
[-]Wei Dai20

so making choices which drop the odds of success so drastically

I wouldn't say "drastically" here so maybe this is the crux. I think the chances of success if China does make an all out push for semiconductors is very low given its own resources and likely US and allies' responses (e.g. they could collectively way outspend China on their own subsidies). I could express this as <1% chance of having a world leading semi fab 10 years from now and <5% chance 20 years from now, no matter what China chooses to do at this point. If hegemony was the only goal then even a 1% chance would be worth it, but "drastically" makes me think maybe that's not what you're saying. These are off the cuff numbers so I'm pretty open to changing my mind about them, but seem reasonable given what I know about China's research capabilities and what it took for the world to reach its current level of semiconductor technology.

Reply
[-]ryan_b81

Separately from gwern's argument, I say that maintaining the gap is still of vital national interest. As an example, one of the arguments in favor of nuclear testing bans is that it unilaterally favors American nuclear supremacy, because only the US has the computational resources to conduct simulations good enough to be used in engineering new weapons.

That logic was applied to Russia, but the same logic applies to China: advanced simulations are useful for almost every dimension of military competition. If they let advanced compute go, that means that the US will be multiple qualitative generations ahead in terms of our ability to simulate, predict, and test-without-risk.

This is a terrible position to be in, geopolitically.

Reply
[-]fylo31

Thanks for really interesting discussion and summary of the state of play. I'm curious what you think of Ben Thompson's point that China could have an advantage at the "trailing edge"(28-45nm, and 45+) if TSMC were taken off the board? 

https://stratechery.com/2022/chips-and-china/

Maybe the play is to focus on the trailing edge, which is easier to take control of, but still v important in the short-term at least, and temporarily let go of the cutting edge?

Reply
[-]Nanda Ale20

Whenever I try and think about Xi's actions as rational I get hung up on the neverending Zero COVID. Many genuinely think it's mostly about saving face but even if I try hard I can't see how it could look anything but childish. They must have convinced themselves it's actually a good policy. I could at least understand that!

Reply
[-]Wei Dai116

I wonder if given the COVID and real estate crises, Xi's government just doesn't have the financial resources to bail out the chips industry, plus maybe they (correctly?) understand that the likelihood of building an internationally competitive chips industry is poor (given the sanctions) even if they do dump in another $200b?

Also, it seems like China is being less antagonistic towards Taiwan and other countries in the last few days. Together with the lack of chips bailout, maybe it means they've realized that it was too early to "go loud" and are pivoting back to a softer, more cooperative strategy.

Reply
[-]aog102

“China is working on a more than 1 trillion yuan ($143 billion) support package for its semiconductor industry.”

“The majority of the financial assistance would be used to subsidise the purchases of domestic semiconductor equipment by Chinese firms, mainly semiconductor fabrication plants, or fabs, they said.”

“Such companies would be entitled to a 20% subsidy on the cost of purchases, the three sources said.”

https://www.reuters.com/technology/china-plans-over-143-bln-push-boost-domestic-chips-compete-with-us-sources-2022-12-13/

My impression is this is too little, too late. Does it change any of your forecasts or analysis?

Reply
[-]Lao Mein10

I have a few questions:

  1. What would you do if all of your investments were stuck in the Chinese market? Puts on semi-conductor-reliant industries?
  2. How much benefit can Chinese firms get from ignoring NVIDIA GPU ML training restrictions? 
  3. Would it be possible to post a few sources?
Reply
[-]aog133

So is this a good thing for AI x-risk? Do you support a broader policy of the US crippling Chinese AI?

Reply
[-]gwern*266

Uh... it's either a very good thing or bad thing. As the Chinese quote goes, it is too soon to tell. If I had to come down on one side, right now, I think I would come down on 'good'; slowing down Chinese AI, which would by default hand AGI to Xi & generally pays even less attention to safety than everyone else does, is good, and while 'CCP destroys TSMC' is far from the ideal approach to restricting compute growth, it is at least going to make a difference - unlike almost all other proposals. The upfront cost in economics, human welfare, peace, rules-based global order etc is going to be exorbitant, however. In my lifetime, wars have not had a good track record of producing solutions at low costs. (The situation reminds me of nukes, Seoul, or EU dependence on Russian gas: it could probably have been prevented cheaply early on with the stroke of a pen, but several decades later...)

Reply
[-]Alex K. Chen (parrot)60

Even if TSMC is destroyed, Samsung has acquired much of TSMC's hard-earned chip-fab/experience curves/tacit knowledge, so scaling in the US should not be set back that much.

(we will have Liang Mong-Song entirely to thank)

Reply
[-]Lao Mein40

Oh, crap, I work in Chinese genetics. If Illumina and other suppliers embargo China, my job will go tits up. Any idea how much BGI Genomics will be affected? I was also considering jumping ship to UAV development, so this has been a really bad month. No one I work with knew any of this was happening when I brought it up earlier this week.

Reply
[-]gwern*60

BGI Genomics, like the mothership in mainland China proper? I'm less familiar with the corporate structure and revenue and dependency on overseas supply chains than I would hope you are, but my understanding was that BGI didn't have much business overseas (having failed to compete with Illumina), and was reliant on domestic demand mostly from agriculture & medicine, and didn't depend on Illumina having spent the past decade+ trying and mostly failing to surpass Illumina (but at least it does have its own decent sequencers). So, having failed in those ways and forced into autarky, it doesn't seem like sanctions/embargos can hurt BGI much more than it already has been? And macroeconomics-wise, sequencing seems like it would be reasonably robust a business because farmers won't stop needing genetics-related services nor will patients stop getting sick. And politics-wise, I see no particular Xi-techlash angle for him doing things like overnight outlawing the industry or censors just refusing to approve any video game release for a year. So overall, doesn't seem too bad.

Honestly, UAV seems like it'd be a worse place to go simply because that sounds to me like it'll be more disrupted by random chip problems and export issues due to being extremely military-linked dual-use tech. Drones are stuffed full of all sorts of random weird little chips (see: Russian problems getting UAVs and resorts like getting them from Iran), and how much domestic Chinese demand for UAVs could there possibly be to make up for exports?

Reply
[-]Lao Mein20

Thanks, I was just really worried because our entire sequencing pipeline uses Illumina products. But I asked around our sequencing division, and they think the difference between using Illumina and BGI products isn't too big - what BGI lacks in quality it makes up for in lower costs. Apparently, the difference in Q30 (% of reads with errors <0.1% of bases) is ~90-95 for Illumina and 80 for BGI, which is marginal. Switching wouldn't be a major problem according to them. BGI also uses Chinese chipsets, which means sanctions aren't going to impact it much. I don't think this is going to be as bad as I first thought.

How likely do you think China getting cut off from Illumina is? Do you think consumer GPUs are going to be restricted?

Reply
[-]gwern*112

Yes, that was my impression. BGI sequencing is not as good as Illumina, but it's not like it'd destroy them to switch over. And sequencers don't use any really high-end GPUs (even if you'd like to have them for bioinformatics), so it's not like a chip embargo is an immediate halt-production problem the way it is for building a new supercomputer or building cars stuffed full of miscellaneous chips

Do you think consumer GPUs are going to be restricted?

Based on a plain reading, consumer GPUs already are restricted: it's not a ban on A100s/H100s, it's a ban on any system as powerful as an A100, to quote Nvidia:

The license requirement also includes any future NVIDIA integrated circuit achieving both peak performance and chip-to-chip I/O performance equal to or greater than thresholds that are roughly equivalent to the A100, as well as any system that includes those circuits.

This is a static threshold, which makes no allowance for Moore's law. It is an upheld hand: "Thus far, and no further!" And it is a threshold which may be biting already. EDIT: it looks like they are going to grandfather in consumer GPUs, ignoring their TFLOPs and focusing only on the interconnect bandwidth+latency, where I think all Nvidia consumer GPUs < A100. This may be a bad bet: people have had not much incentive to figure out how to yoke together large numbers of consumer GPUs (as most people using >1 consumer GPU are either hobbyists who generally can stick all the GPUs they can afford into a single box, or cryptominers for whom interconnect is irrelevant because the PoW mining is by design embarassingly-parallel), so if Chinese hyperscalers or major AI labs can purchase all the 4090s they want, now there will be vastly more incentive to figure out hardware hacks (will, say, TSMC ban Chinese designs for interconnect chips which do no computation...?) or low-communication strategies (even large constant factor penalties, which would render them useless to Western groups, will eventually be worthwhile to Chinese groups barred from >=A100 GPUs.)

What is the 'peak performance' roughly equivalent to an A100? Well, Nvidia's product page tells me an A100 delivers 19 FP32 TFLOPs, where FP32 is probably a conservative number. (I don't think there are very many cases in DL where you would still need FP64, and while everyone is trying to move to FP16 and lower, where an A100 does 312 TFLOPS instead, you still can't always easily or reliably train every arch with FP/BF16 and you can encounter some severe problems especially with the important case of large Transformers.) 19 TFLOPS may sound like a lot... but the top-end gaming GPU Nvidia RTX 4090 goes as high as 86 TFLOPS (specced at 82 TFLOPS, with GeForce RTX 3090 Ti at 40 TFLOPS and GeForce RTX 2080 Ti at 14 TFLOPS).

(The H100, incidentally, is 67 TFLOPS.)

So, if USG is banning 'peak performance roughly equivalent to A100', and gaming GPUs are turning in TFLOPS several times that of the A100, then the obvious interpretation would seem to be that yeah, lots of consumer GPUs are already restricted. All the way back to like 2019 GPUs, possibly.

(I was really surprised by this when I went to check the numbers; I did not think that gaming GPUs were that much faster than the A100. I knew that the A100 was already kinda obsolete with the H100 in the pipeline, that there was a 'datacenter tax' and 'enterprise tax', and also that everyone has kinda understated GPU progress because they were so hard to get for so long and most people have been trucking along with GPUs years out of date - even the V100 still shows up in papers occasionally - etc, but I didn't think it would be that much slower, like 4x. Either I am badly misunderstanding units here, or I am greatly overrating A100s & underrating recent gaming GPUs like the absurdly large 4090, or I am making a mistake in looking at FP32 because Nvidia put all of the A100 advantages into lower precisions.)

Reply
[-]gwern70

https://twitter.com/paul_scharre/status/1585684356819046400

Under Secretary of Commerce Alan Estevez confirmed at a public @CNASdc event this morning [2022-10-27] that Commerce intends to keep the same technical threshold for chips in place over time.

Reply
[-]aog10

Why are gaming GPUs faster than ML GPUs? Are the two somehow adapted to their special purposes, or should ML people just be using gaming GPUs?

Reply
[-]Nanda Ale*70

>Why are gaming GPUs faster than ML GPUs? Are the two somehow adapted to their special purposes, or should ML people just be using gaming GPUs?

They aren't really that much faster, they are basically the same chips. It's just that the pro version is 4X as expensive. It's mostly a datacenter tax. The gaming GPUs do generally run way hotter and power hungry, especially boosting higher, and this puts them ahead against the equivalent ML GPUs in some scenarios.

Price difference is not only a tax though - the ML GPUs do have differences but it usually swings things by 10 to 30 percent, occasionally more. Additionally the pro versions typically have 2X-4X the GPU memory which is a huge qualitative difference in capability, and they are physically smaller and run cooler so you can put a bunch of them inside one server and link them together with high speed NVLink cables into configurations that aren't practical with 3090s. 3090s have a single NVLink port. Pro cards have three. 4090s curiously have zero - NVIDIA likely trying to stop the trend of using cheap gaming GPUs for research. Also, the ML GPUs for last gen were also on a a 7nm TSMC process, while the gaming GPUs were on Samsung 8nm process. This means the A100 using 250 watts outperforms the 3090 using 400 watts. But they are overall the same chip. 

None of that accounts for a 4X or more cost multiplier, and the TLDR is the chips are not that different. If gaming GPUs came in higher memory configurations, and all supported NVLink, and were legally allowed to be sold in datacenters, nobody would pay the cost multiplier.

Reply
[-]Lao Mein00

I just realized that H100s are still available from online vendors for ~189000 yuan (~$27000), which is international market price.

Well then. Time to cash out my savings and make some money. Do you guys think it's feasible? How much are Chinese H100 prices likely to rise? Should I be trying to scoop up high-end GPUs instead?

Update: local computer shop says international GPU suppliers are still accepting Chinese orders and Chinese GPU prices are stable for now.

Reply
[-]aog10

China's top chipmaker, Semiconductor Manufacturing International Corporation, built a 7nm chip in July of this year. This was a big improvement on their previous best of 14nm and apparently surpassed expectations, though is still well behind IBM's 2nm chip and other similarly small ones. This was particularly surprising given that the Trump administration blacklisted SMIC two years ago, meaning for years they've been subject to similar restrictions as Biden recently imposed on all of China. Tough to put this in the right context, but we should follow progress of China's chipmakers going forwards. 

Reply
[-]trevor103

in preparation for another Cold War

Sorry to nitpick, but this is a pretty big snag when it comes to World Modelling for most people. The original Cold War started in the wake of WW2, by all the people who oversaw WW2 and were getting promotions heaped on top of them due to their contributions.

The important thing is that 1) the Cold War was started with extremely similar mentalities and people as WW2, and 2) it was started with intent to nuke (carpet bombing metropolitan areas was considered standard military doctrine during WW2). The mentalities and doctrines that sculpted the Cold War were replaced before the Cold War even ended. So calling it "another Cold War" is epistemically harmful because it prescribes a long list of features (e.g. elites believing that diplomacy will inevitably fail and that carpet bombing will be with nukes), and any actual conflict in the modern world will never, ever check the requirements needed in order to resemble the environment of the Cold War. So the idea of "either we go back to the Cold War or we don't" is a misleading misnomer. The world can definitely share some features with 1950-1990, but it will never share most of the features with that time period.

Reply
[-]RHollerith*20

Huh. I always thought that if we were magically/hypothetically given the option to limit the death and destruction caused by the next drawn-out competition among great powers to that caused by the Cold War (which includes of course the Korean War, the Vietnam War and the Soviet occupation of Afghanistan) we should jump at the chance to choose that option.

I always thought that the Cold War turned out remarkably benignly given the weapons available to the warring sides and given previous conflicts between great powers.

Do I read you correctly as believing that humanity has entered a more enlightened era where outcomes as bad as what happened to people who got caught up in the Cold War are unlikely?

Reply
[-]quanticle20

The important thing is that 1) the Cold War was started with extremely similar mentalities and people as WW2, and 2) it was started with intent to nuke (carpet bombing metropolitan areas was considered standard military doctrine during WW2).

Do you have any citations for those claims? Dean Acheson's book, Present at the Creation is a memoir of the early days of the Cold War, and shows that, at least in the US, policymakers dropped the intent to nuke pretty quickly. Indeed, George Kennan wrote his famous long telegram, outlining the strategy of containment, in 1946.

Yes, there were hotheads like MacArthur, who advocated total (thermonuclear) war in order to wipe Communism off the face of the Earth, but they were a small minority. Vocal, but small.

Reply
[-]aog12

Yeah I basically agree. My intention was a period of heightened tensions and overt conflict between the China and the US, not usually escalating into direct combat, but involving much of the world and using most of the available political, economic, and military leverage. The belief in nuclear armageddon seems like the biggest difference between this time and the original Cold War. Perhaps there’s also less ideological difference, though I’m not sure about this.

Reply
[-]trevor71

I agree that wording it as "It's part of a larger strategy of economic decoupling from China in preparation for another Cold War" was really helpful and went a long way to give me a sense of why decoupling is being taken so seriously, and I have no idea how to word it better. 

I also think that decoupling is often seen by many as the big factor that distinguishes the current Cold War from the old one, but that might be popular bias, since lots of people are paid to analyze that whereas both nuclear beliefs and democracy-versus-authoritarian ideologies involve tons of black box institutions and military secrets.

Reply
[-]Lao Mein50

Well, this is awkward. Any advice for a Chinese bioinformatics coder interested in finding a well-paying job in the US? I want to go into AI research and went to college in the US if that helps.

Reply
[-]Lone Pine41

I wonder if this move could end NVIDIA's dominance in ML, if Chinese AI research is forced to move to different platforms. If there is a need, the needed software tooling will appear quickly.

China has put an extraordinary amount of resources into developing their own chip fabrication capacities, and so far has failed pretty pathetically. But they will continue to try, and if these sanctions continue, they will presumably be more successful (due to a large captive domestic market).

The Asianometery YouTube channel has done a lot of great reporting on the semiconductor industry in China and elsewhere.

Reply
[-]RHollerith110

China has put an extraordinary amount of resources into developing their own chip fabrication capacities, and so far has failed pretty pathetically and now IMO it is probably too late: China will more likely than not never acquire the capability to design and fab GPUs and TPUs as efficient as those currently being designed by Nvidia and Google respectively and currently being fabbed by TSMC and whoever is fabbing Google's TPUs.

The ability to design and fab such chips is currently a "world ability": the fabbing relies on litho machines made by in the Netherlands, which in turn depends heavily on lenses made by Zeiss in Germany and other advanced products from Japan, the US and probably other countries. The fabbing relies on very expensive other products that like the litho machines are made by just a handful of companies, almost all of which are vulnerable to pressure from the US government (especially if other governments generally aligned with the US tend to agree with the USGov's reasons). Designing the chips (the layouts) requires software available from only a handful of suppliers again probably all vulnerable to pressure from the USGov. It is currently a world ability and more likely than not will remain a world ability and not an ability of any one bloc (e.g., the Chinese government and its friends) unless we consider the US and Europe and entities vulnerable to pressure from those 2 as one bloc.

Note the "more likely than not" above! I'm not saying that I can reliably predict the relative strength of China in the future! It's just that I am not moved by the numerous pronouncements that the rise of China is inevitable. To take just one example, Peter Zeihan says that if China were subjected to the same sanctions imposed on Russia after Feb 24 of this year, there would be rolling blackouts within a few months, trucks running out of gas and abandoned on its highways and in about a year half of the population will have died of starvation: in contrast to Russia, which produces more fossil fuels, fertilizer and basic foodstuffs like wheat than it needs, China is heavily dependent on imports for those things.

Zeihan goes on to say that although China has always been aware of the possibility that US and Europe governments would impose sanctions on it, it was utterly shocked to learn of the possibility that major corporations would in the future refuse to do business with China without even being forced to by a government. He says he has heard from sources inside China that such an eventuality was never considered by Chinese planners and now they feel they must re-evaluate their entire geopolitical strategy.

There is a danger when spending a lot of time reading comments on the internet -- or when reading the output of our newspapers and similar institutions -- to allow the sheer repetition of a claim thousands of times from thousands of individual authors to give the claim more credence than it deserves. The vast majority of the claims I have seen as to the probable eclipsing of the US by China either makes no argument in support of the claim or the included argument is very unimpressive. Yes, there are experts I respect, like John Mearsheimer, who make the claim (though I haven't heard him make an argument for it: he just claims it, but he is Mearsheimer, so I listen), but there are other geopolitical experts I respect, like George Friedman, who believe that China will be lucky if 15 years from now it hasn't split into 2 or 3 countries -- and that the Chinese government knows that and preventing it is their main focus. Friedman also says that the Chinese navy is still not capable enough to protect the ships that bring it oil from the Persian Gulf from pirates and privateers if the US navy stepped down from its role of protecting the world's shipping from pirates. "Privateers": if there ever is a really serious oil shortage, the Indian government might follow a centuries-old tradition of defining the takeover of oil tankers bound to China as "piracy", but rather "privateering", and consequently not illegal and in fact maybe it is rewarded, and according to Friedman, there is nothing the Chinese navy can do about that: they simply lack the ability to project force that far from port. Friedman also says that the Japanese navy would more likely than not prevail over the Chinese navy if they fought today. He says that that is partly because Japan's admirals were trained by admirals who were trained by admirals who actually fought a major war with what remains the dominant naval technology relevant to a major war, namely, the aircraft carrier -- and according to him, that that counts for a lot.

The reason I'm commenting is that this has relevance to managing the risk of AI: in particular, in a situation as dire as the current global situation around AI research, we should be paying attention to those futures in which our ability to control and manage the global situation around AI has drastically increased even if right now those futures seem unlikely. In particular, there is a chance that the leaders of China will in the future do something at least as risky and ill-fated as Putin's decision to invade Ukraine, and the response to that by the West will make it so that we don't have to worry about AI researchers in China because China will be too preoccupied with feeding itself and just keeping the country supplied with electricity and whatever China uses to heat its homes. Or maybe Southern China or Shanghai will make a serious attempt to secede from Beijing's rule, in response to which Beijing will lock the country down kinda like they did for Covid, with the same effects on the Chinese economy that the Covid lock-downs had, but these lock-downs will go on for years, and again then we don't have to worry about the danger posed by Chinese AI researchers.

So if young people concerned about AI risk move to Washington and London and Paris and start a career in the national-security bureaucracy there with the goal of eventually convincing their government to impose a regime of "compute governance" similar to the existing regime of nuclear non-proliferation, then maybe their efforts will be in vain because the center of mass of irresponsible AI research (currently somewhere in the Atlantic between the US and Britain) will just move to China. But maybe their efforts won't be in vain because China will stop being able to host the intensity of scientific research and technological development necessary to contribute significantly to the AI danger. And maybe Russia will continue its decline with the result that regardless of the wishes and the ambitions of its government, it becomes unable to contribute significantly to the AI danger (which of course its government does not perceive as particularly dangerous).

Reply
[-]ChristianKl80

The vast majority of the claims I have seen as to the probable eclipsing of the US by China either makes no argument in support of the claim or the included argument is very unimpressive.

The main argument for increased strength is that China manages to keep a high growth rate. Last year they managed to get back to 8 percent. Increased money also means increased foreign investments and thus more ways to project power.

And maybe Russia will continue its decline with the result that regardless of the wishes and the ambitions of its government, it becomes unable to contribute significantly to the AI danger (which of course its government does not perceive as particularly dangerous).

The whole rationality community in Sanct Petersburg left Russia. Between braindrain, embargos to import chips, and corrupt corporate governance I don't expect Russia to be able to do much in the field of AI.

I would expect more risk from a country like Argentina that currently leads in zero-day exploit generation than from Russia. 

Reply
[-]RHollerith*72

During the Cold War, no one believed the economic figures published by the Soviet Union. I am puzzled as to why the figures published by the Chinese Communist party are often taken at face value now. Yes, I concede that the average Chinese resident really is wealthier than the average resident of India. But I don't necessarily believe the GDP figures that say he or she is wealthier than the average Mexican, just to pick a country for the sake of illustration that according to the World Bank has a GDP per capita a little lower than China's. And I'm much less certain than "the internet" that the average Chinese person will ever be wealthier than the average Mexican since Mexico is much less likely to descend into political chaos than China is and since Mexico has significant petroleum reserves and access to cheap Texan natural gas.

The main argument for the potential of China IMO is the staggering number of people with IQs over 130, many of whom have degrees in science or engineering. But whereas scientific and engineering knowledge transfer fairly easily to any sufficiently bright population anywhere in the world, the ability to organize a society such that the scientists and engineers can contribute to societal goals is not so easy, and Russia is a good example of this because although Russia certainly can produce great scientists (just restricting ourselves to the 20th Century, we find Solomonoff, Kolmogorov and Grigori Perelman) the society is lacking in many ways (e.g., the quality of life of the average person is low; e.g., even if you're wealthy, the chances of your being treated unjustly by the government or the justice system is much higher than it is in many countries; e.g., Russia does not contribute to the flourishing of the world anywhere near as much as the West does).

Again, I am not making a prediction about how powerful China will be 10 or 20 years from now; I'm merely saying that I'm much less certain that it will continue to grow in capability than the average internet commentator is.

Reply
[-]ChristianKl40

Absolute GDP values aren't very trustworthy just like the absolute COVID numbers that China publishes are not trustworthy. On the other hand, it's hard to fake exponential growth over longer timeframes. 

But whereas scientific and engineering knowledge transfer fairly easily to any sufficiently bright population anywhere in the world, the ability to organize a society such that the scientists and engineers can contribute to societal goals is not so easy, and Russia is a good example

Russia mainly exports resources and not technology. They indeed didn't manage to organize their society in a way that allows engineers to contribute effectively. 

China is different. They manage to export a lot of technology. They are the most efficient producer of solar cells. 

If we look at their domestic market, they build better trains than the West. They have better digital payment. 

Baidu just won a license to start offering fully autonomous robotaxis in some test cities. I think it's plausible that China is better than the West in transferring engineering knowledge to products because their government is more willing to get regulatory barriers out of the way than Western governments are. 

China also has a good startup culture with lots of Unicorns. 

Reply
[-]Zac Hatfield-Dodds72

The relevant tooling already exists: https://github.com/THUDM/CodeGeeX is a 13b param model which would have been SOTA two years ago, trained on Chinese hardware, using a Chinese deep learning framework.

Anyone who thinks that compute restrictions will help with x-risk should explain how they plan to convince China that this isn't just a geopolitical ploy by the US to cripple their advanced industry and related military capabilities. (despite, you know, that being the motivation for export controls to date)

Reply
[-]Zac Hatfield-Dodds50

Yep, this is a bigger deal than I realized last week.

Reply
[-]jacob_cannell20

Wow - if those stats are correct, the training of CodeGeeX used up to 1e24 nominal flops (2.56e14 flops/chip * 1536 chips * 2.6e6 seconds), which would put it a bit ahead of Chinchilla, although its seemingly lite on param count. But it is somewhat easier to tile a chip with fp16 units then it is to utilize them effectively, so the true useful flops may be lower.

Nonetheless, that's quite surprising, impressive, and perhaps concerning.

Reply
[-]Zac Hatfield-Dodds10

Distributed training runs never manage to fully utilize nominal flops from hardware, and are easy to stuff up in other ways too, but I'd expect the chips themselves to be pretty well set out - it's obvious early in the design stage if you're going to be bottlenecked on something else.

Reply
[-]aog11

I’m open to the idea that this is good from an x-risk perspective but definitely not 100% sold on it. I agree with you that China knows this is directly aiming to cripple their advanced industry and related military capabilities. We’re entering an era of open hostilities and that’s not news to anyone on either side. I don’t think that necessarily means this is bad from an x-risk perspective.

A few claims relevant to analyzing x-risk in the context of US-China policy:

  1. AGI alignment is more likely if AGI is developed by US groups rather than Chinese groups, because some influential people in the US take alignment seriously while almost nobody in China does.

  2. Slowing Chinese AI progress is good because it makes the US more likely to be the first to AGI, or gives more time for the necessary alignment work to happen and spread to China.

  3. US policy can actually slow Chinese AI development with export bans on compute.

I currently believe these three claims. CodeGeeX is pretty good evidence against 3, showing that Chinese compute and tooling is catching up to that of the US. But building high performance compute is a complicated supply chain with a lot of choke points, and it seems plausible that the US can slow Chinese progress by a few years. See e.g. this CSET article: https://cset.georgetown.edu/wp-content/uploads/Preserving-the-Chokepoints.pdf

IMO the strongest argument against this policy from an x-risk perspective is that this reduces future influence over Chinese AI development by using a one-time slowdown right now. If the most critical time for slowing AI progress is in the future, this bullet will no longer be in the chamber. But I also haven’t spent much time thinking about this and would welcome better arguments.

Reply
[-]aog10

I was wrong that nobody in China takes alignment seriously! Concordia Consulting led by Brian Tse and Tianxia seem to be leading the charge. See this post and specifically this comment. To the degree that poor US-China relations slow the spread of alignment work in China, current US policy seems harmful. 

Reply
[-]Zac Hatfield-Dodds10

I think that convincing Chinese researchers and policy-makers of the importance of the alignment problem would be very valuable, but it also risks changing the focus of race dynamics to AGI, and is therefore very risky. The last thing you want to do is leave the CCP convinced that AGI is very important but safety isn't, as happened to John Carmack! Also beware thinking you're in a race.

I think (3) is only true over timescales of 2--5 years: the thing that really matters is performance per unit cost, and if you own the manufacturer you're not paying Nvidia's ~80% unit margins.

Reply
[-]Noosphere8910

I think that convincing Chinese researchers and policy-makers of the importance of the alignment problem would be very valuable, but it also risks changing the focus of race dynamics to AGI, and is therefore very risky. The last thing you want to do is leave the CCP convinced that AGI is very important but safety isn't, as happened to John Carmack! Also beware thinking you're in a race.

Unfortunately for us LessWrongers, it's probably rational for them to believe that a race is happening, because to put it bluntly, China having AGI would become the world power that exceeds even the US. Problem is, the US government doesn't care about the alignment problem.

Reply
[-]jacob_cannell73

US foundries (intel + IBM/AMD/Gobal) are also behind TSMC, china is way behind. Last estimates I heard was that intel was about 4 years behind TSMC, and I can only assume china is even farther behind. 6 years is an enormous gap when perf is improving 100% per year (which is the recent pace of Nvidia/TSMC) .. but if Moore's Law ends in the next node or two then suddenly the leaders hit a wall and the rest start to catch up. But even just a few years lead should give the US/west a big edge in reaching AGI first.

Reply
[-]KingOfMadPistoleros20

Quick question: isn't the US depending on China for most or all of its rare earth processing ?

Reply
[-]cubefox2-5

Given that this is an act of economic warfare, how does the US government justify it? For liberal western democracies, "it is in our selfish interest to do so" is typically not seen as a sufficient reason. Usually there is some justification of why it is the morally right thing to do. I'm concerned this part is missing here.

Reply
[-]ChristianKl0-2

The justification is that there's a risk that the Chinese military will use it. It seems to be part of general export controls for military hardware.

"This includes preventing China's acquisition and use of US technology in the context of its military-civil fusion program to fuel its military modernisation efforts, conduct human rights abuses, and enable other malign activities."

Reply
[-]cubefox*20

The linked article is more than a week old, though, it appears not to talk exactly about the same thing. Moreover, presumably every military in the world has to use microchips, so a China specific ban seems to require further justification.

Reply
[-]M. Y. Zuo1-1

Some of the companies that are specifically targeted seem particularly heinous such as SenseTime and Dahua which provide facial recognition software for the concentration camps in Xinjiang, 

Haven't the claims of 'concentration camps' been discredited? 

Last time I checked there's only solid evidence for penal labor related abuses like any penal labor system, plus possibly systematic discrimination due to 'de-radicalization'.

The former is pretty normal in the penal system of any developing country's poorer regions such as Malaysia, Indonesia, Brazil, etc. 

The latter claims might have some merit yet the shadiness of the proponents really make it difficult to believe.

For example, there's a lot of evidence that many people boosting these claims are ideologically/religously motivated. It also didn't help that some of these folks, including major media outlets, backed some provably false claims and presented stuff that turned out to be fake.

Reply
[-]Randaly41

No.

I'm pretty negative on how you fail to discuss any specific claim or link to any specific evidence, but you spend your longest paragraph speculating about the supposed bias of unnamed people.

You haven't really written enough to be clear, but I suspect that you have confused concentration camps with death or extermination camps? Regardless, the recent UN report did pretty specifically support claims of concentration camps- see points 37-57

Reply
[-]M. Y. Zuo-20

The biases I'm referring to are not 'supposed' they are openly advertised by the same proponents, it's especially obvious in the case of certain religious fundamentalist groups.

Reply
Moderation Log
More from aog
View more
Curated and popular this week
58Comments