The geopolitics of AI
Chips are an essential cog in the wheels of a digital economy, without which advanced industries would be grounded. All of the promises of AI are dependent on the most advanced semiconductor technology, design, and fabrication. About 75% of global semiconductor manufacturing capacity is concentrated in China and East Asia, a region significantly exposed to high seismic activity and geopolitical tensions. In addition, 100% of the world’s most advanced (below 10 nanometers) semiconductor manufacturing capacity is currently located in Taiwan (92%) and South Korea (8%). These advanced chips are essential to America’s economy, national security, and critical infrastructure.
The U.S. is completely dependent on semiconductor fabrication facilities in Taiwan for the most advanced chips. Without companies located in Taiwan, a country China covets, the U.S. preeminent position in AI would collapse. These are potential single points of failure that could be disrupted by natural disasters, infrastructure shutdowns, or international conflicts, and may cause severe interruptions in the supply of essential chips.
The US is trying to cut off China’s ability to make advanced semiconductors, on the judgment that advanced semiconductors are critical to training AI systems. Taiwan Semiconductor will not begin to fabricate advanced semiconductors at its new Arizona facility until the end of 2024, a lifetime in the semiconductor space. Construction of advanced semiconductor facilities is being delayed because semiconductor companies resist the conditions imposed by the Biden administration on the release of funds from the CHIPS Act. Morris Chang, the founder of Taiwan Semiconductor, has already stated that the U.S. will never be a low-cost producer of advanced semiconductors. It is too difficult to build in the U.S., and U.S. labor is not cost-competitive. Taiwan Semiconductor invests about $35 billion annually in its semiconductor facilities. Samsung is investing $230 billion to build one new advanced semiconductor manufacturing facility. The $53 billion appropriated domestic manufacturing, and workforce development is chump change compared to the investments of Taiwan Semiconductor and Samsung. Until the U.S. dramatically lowers the cost of capital for the semiconductor industry, dramatically streamlines the permitting process, and dramatically improves labor productivity, the U.S. will never be able to compete against Taiwan or South Korea in the advanced fabrication of semiconductors.
To make an advanced semiconductor, you need to buy machine tools from just a handful of companies around the world that have the precision capabilities to manufacture these tools. One of the most important of these companies is a dutch firm called ASML, based in the Netherlands. It has unique capabilities — which no one else in the world can replicate — to produce a type of machine called an EUV lithography tool, without which making an advanced chip is simply impossible.
The US Commerce Department added NVIDIA’s powerful AI chips, the A100 and H100, to its export control list. NVIDIA later announced the development of the A800 and H800 chips, designed to meet the performance threshold just below the US ban. Chinese tech giants, Tencent and Alibaba have already purchased large quantities of these chips. To bypass restrictions, Chinese companies have resorted to renting cloud computing resources. Reuters reported conversations with vendors in Shenzhen’s Huaqiangbei electronics area who could supply A100 chips, albeit without warranty or support. The demand for these chips extends beyond AI-focused companies and includes app developers and gamers affected by the AI chip ban. There are claims that this chip traffic benefits local authorities. The CEO of Megvii, a Beijing-based face recognition software maker, estimated that there are approximately 40,000 A100 GPUs in China. China has historically accounted for 20–25% of Nvidia’s revenue from data center products, although the more advanced H100 chips remain out of reach. While small quantities of A100 chips are relatively accessible, the export of larger batches appears to be under control. According to the Chinese website Late Post, ByteDance has reportedly ordered over $1 billion worth of GPUs from NVIDIA, likely referring to the A800/H800 chips.
Their decision to stock up on GPUs has proven beneficial. However, the situation might worsen, potentially as early as this summer. The Wall Street Journal reports that the Biden administration is considering a ban on selling A800 chips without a license as part of updated export controls. Building a high-quality chatbot like ChatGPT requires a substantial number of GPUs. Baidu, however, seems to possess an ample supply as it recently claimed that its chatbot, Ernie 3.5, outperformed GPT-3.5 on two LLM benchmarks: AGIEval (an exam-heavy benchmark created by Microsoft) and C-Eval (a Chinese-focused benchmark developed by Shanghai Jiao Tong University, Tsinghua University, and the University of Edinburgh). Additionally, Ernie 3.5 surpassed GPT-4 in Chinese-language capabilities. Baidu has emerged as a leading player in LLM development within the Chinese industry and hopes to reap the rewards of years of research. Moreover, Baidu may not face foreign competition in Hong Kong since Microsoft/OpenAI and Google are restricting access to their chatbots, likely due to concerns about generating content that violates Hong Kong’s strict new national security law.
All the current demand and expected growth of AI use has fueled NVIDIA’s valuation during the past few weeks. Who’s to compete with them and to benefit from the supply crunch? AMD? George Hotz a security hacker founded a new company (tinygrad) to battle CUDA/ NVIDIA. AMD announced two partnerships with Pytorch and Huggingface. The first aims at making Pytorch immediately compatible with ROCm (AMD’s CUDA) and running training jobs on AMD’s latest MI300 GPU. The second enters AMD in HuggingFace’s Hardware Partner Program, where developers from both companies will work to make models from the HuggingFace hub run on AMD’s chips. AMD’s latest chips probably lag behind NVIDIA’s in multi-server settings. More importantly, the partnerships, or tinygrad, won’t solve AMD’s software woes any time soon.
EU can’t be sitting with so much happening. They approved the draft of the AI act this month with rules specifically designed to regulate the use of foundational models. Nice work from Stanford’s Center for Research on Foundation Models graded the foundation model provider’s according to their compliance with the Draft EU AI Act. They used the 12 draft AI criteria that could be measured using public information and graded the providers on each on a scale of 0 to 4. The criteria included the use of copyrighted data (on which all but HuggingFace/BigScience’s BLOOM (3) and GPT-NeoX (4) scored 0), energy consumption, documentation, and risk mitigation. BLOOM came out on top with a score of 36/48, while Anthropic’s Claude was last with 7/48. But more than the scores, the researchers highlighted that a compliant foundation model isn’t totally out of reach, as taking the pointwise maximum between BLOOM and GPT-4 over each criterion resulted in a 42 score, i.e. satisfying 90% of the criteria. The achievability of compliance has not, however, stopped a range of prominent EU companies warning that the Act could have a serious impact on competitiveness and technological sovereignty. A letter signed by 150, including European champions like Heineken, Siemens, and Renault, warned that the AI Act risked making compliance prohibitively expensive, and instead advocated a risk-based approach.
While the EU took its AI Act victory lap, the UK attempted to get back on the front foot. Firstly, the UK announced plans to be the power broker and host a major summit for “like-minded” nations (read: not Russia or China) on the future of AI. The agenda for this summit remains unclear, but it seems as though the government is attempting to revive its traditional position as a “bridge” between divergent EU and US regulatory regimes.
It remains to be seen, how the UK’s aspirations around global leadership will shape up. The Prime Minister revealed his safe strategy for AI, which included encouraging steps around safety, such as a commitment from DeepMind, OpenAI, and Anthropic to provide early access to frontier models for AI safety research.
In the world of merchant silicon, Intel, is nowhere to be found despite acquisitions of 2 different datacenter AI hardware firms, Nervana and Habana. Nervana was killed a few years ago, and the same seems to be happening to Habana now. Intel is currently on their 2nd generation Habana Gaudi 2 with little to no adoption besides some instances available on AWS. Furthermore, Intel is already communicating the roadmap as dead with the product being rolled into the 2025 Falcon Shores GPU. Intel’s GPU, Ponte Vecchio isn’t faring any better. It is quite late, having only recently completed delivery to the long-delayed Aurora supercomputer, with no successor for another 2 years. It’s performance is generally uncompetitive with Nvidia’s H100 GPU. Cerebras is currently the closest competitor with solid performance on GPT-3 and impressive open-source models, but hardware accessibility is very limited with each individual server costs millions of dollars. The only way to access Cerebras in the cloud is through their own offering. The lack of access hurts development flexibility. The life blood of the Nvidia ecosystem is people developing on a wide variety systems, from their gaming GPU that costs hundreds of dollars to being able to eventually scale to systems with tens of thousands of GPUs on premises or with all 3rd party cloud service providers. While other startups such as Tenstorrent show promise, we believe the hardware/software is still a bit away from really hitting its stride.
The rise of the cloud, 5G deployment, connected vehicles, and the digitization of everything else have created unprecedented demand for high-performance computing. There’s a serious shortage of semiconductor supply. And foundries are raising prices by as much as 30 percent, amassing huge profits along the way.
As the backbone of the connected world, the semiconductor industry must maintain a sustainable balance of demand and supply. Companies can effectively manage the volatility of the business cycle through step-change improvements in the resiliency and flexibility of their semiconductor supply chain. New approaches are required to strengthen global cooperation on the governance of new technologies, including artificial intelligence (AI) and big data.
We need research on strategies and policies to improve supply chain resilience that would be robust enough to survive the onslaught of economic nationalism in the United States and China, as well as in Europe, Japan and the rest of Asia. Instead of each country trying to become self-sufficient through zero-sum competition and beggar-thy-neighbour policies, a better way to deal with supply chain vulnerabilities would be to negotiate international arrangements, whether through the World Trade Organization (WTO), trade associations, open source or other NGO communities.
US supply chain controls are also damaging leading foreign suppliers in US partner countries, such as Taiwan’s TSMC or ASML in the Netherlands, which are losing important Chinese customers. Sooner or later, these foreign suppliers will find ways to circumvent the US origin restrictions. And they will attempt to design out US content altogether. Each of these foreign suppliers has its own mini-supply chains. Take ASML, the only global supplier of the critically important extreme ultraviolet (EUV) lithography system. With more than 100,000 components, an EUV lithography system is one of the most complex machines ever built, each unit costing more than $120 million. More than 1,000 suppliers are involved, each of them potentially exposed to collateral damage. Of particular importance are the suppliers of EUV’s key components: the Germany-based Trumpf Group, a large multinational technology leader in advanced laser technology; and the Zeiss Group, a global leader in lithography optics. So, to the degree that ASML is damaged by US supply chain controls, Trumpf and Zeiss are equally damaged. It would certainly be unrealistic to expect the IT industry to agree on a joint approach to the current semiconductor supply crisis. Each of these companies treat the supply chain knowledge and processes within their enterprise as competitive weapons. For these companies, supply chain knowledge is critical to satisfy profit expectations of investors and financial analysts.
Finally, what role could open-source communities play? In fact, research and standard-setting for semiconductors now increasingly takes place in open-source communities, such as the Linux Foundation and the RISC-V Foundation, as well as GitHub and many others. These communities tend to avoid international organizations based on state representation, because of fear that geopolitics may disrupt knowledge exchange.
In the end, however, it is unclear whether the quest for improved supply chain resilience will mobilize enough forces to shift the focus of US policy away from supply chain regulation in the service of geopolitics. Too powerful is the cross-party consensus in US Congress that China now poses an existential threat to US leadership in advanced technology, and that this will erode America’s security and military strength. At the same time, the vicious circle of US sanctions and Chinese countermeasures seems to have silenced voices for reconciliation in both countries.
Credits: Excerpts from Nathan Benaich’s July Newsletter
