The Hyperscaler Custom Silicon Wave

By Nekud October 16, 2026 19 min read

In October 2026, a short, sharply argued piece on Axios carried the headline that summarized a strategic shift the AI infrastructure industry had been working toward for the better part of a decade. “Every major Nvidia customer is now a competitor,” the piece reported, and the phrase became, in the days that followed, the framing for the most consequential structural change in the AI compute industry since the emergence of the GPU itself. The piece was not exaggerating. Microsoft had, the same week, unveiled Maia 300. Google was already shipping TPU v8 in volume. Amazon’s Trainium business had crossed a 25-billion-dollar annual revenue run rate. OpenAI was working on Jalapeño with Broadcom. Meta was moving its Iris chip from tape-out to production with the same Broadcom foundry. Anthropic had begun exploring its own chip designs. Even Apple, which does not rent its silicon to anyone, had confirmed that its internal AI acceleration stack continued to incorporate growing amounts of custom hardware. The pattern was unmistakable. The customers of the merchant GPU market, the hyperscalers, the consumer-AI labs, and the largest neocloud operators had collectively decided that the most consequential component of their cost structure would, in the second half of the 2020s, be designed in-house.

The custom-silicon wave is the single most important industry-side development of the AI infrastructure era, and it is the development that is most likely to define what the AI industry looks like in the late 2020s. The reason is structural. The cost of running an AI service at scale is, in late 2026, dominated by the cost of inference silicon and the cost of the electricity that powers it. The cost of inference silicon is set, today, primarily by the merchant GPU market and Nvidia’s pricing within it. The cost of electricity is set by the electricity market. The only variable cost component that a hyperscaler can substantially influence is the cost of inference silicon, and the only way to substantially influence that cost is to design and build silicon that, for the relevant workload, is more cost-effective than the merchant alternatives. That is why every major Nvidia customer has, by late 2026, an active custom-silicon program. The economics demanded it.

This essay traces the custom-silicon wave from its mid-2010s origins in Google’s TPU program through the mid-2020s generalization across the hyperscaler industry. It focuses on four threads. The first is the Google TPU program, which has been running for roughly a decade and which is the only hyperscaler custom-silicon program to have already been through several full design generations. The second is the Microsoft Maia program, which is the most recent entrant and which, in its third-generation Maia 300 design, has produced the most architecturally distinctive inference silicon of the late 2020s. The third is the Amazon Trainium and Inferentia programs, which together account for the largest deployed base of hyperscaler custom inference silicon. The fourth is the OpenAI, Meta, Anthropic, and Qualcomm-Amazon programs, each of which represent a different architectural point in the same wave. The thesis the essay develops is the following. The custom-silicon wave is not a fashion; it is the structural response of the largest AI compute consumers to a cost structure that has become intolerable, and it is going to be a permanent feature of the AI infrastructure industry for the rest of the decade.

The Google TPU decade

Google’s Tensor Processing Unit is the oldest and most architecturally mature of the hyperscaler custom-silicon programs. The first TPU was a 28-nanometer application-specific integrated circuit announced in 2016 and deployed inside Google’s search and machine-learning infrastructure from 2015 onward. The chip was, in retrospect, an early proof that a hyperscaler could build a fully custom AI accelerator and integrate it into a production inference fleet at scale. The TPU was not, in 2015 or 2016, a training chip; it was an inference chip, and its principal workload was the ranking and language-understanding models that powered Google Search and Google Translate. The architectural choices in the original TPU — a large systolic array for matrix multiplication, modest on-chip memory, and a host CPU that streamed data to the accelerator — established the design language that has shaped every subsequent generation of TPU.

The second-generation TPU, announced in 2017, was the first generation to be available to Google’s public-cloud customers as a cloud-rental product. The second-generation chip supported both training and inference, introduced high-bandwidth memory in place of the on-chip SRAM of the first-generation design, and ran, in Google’s published benchmarks, at between 15 and 30 times the throughput per dollar of contemporary CPUs and GPUs on standard machine-learning benchmarks. The second-generation chip was also the first generation on which Google’s own research teams began training substantial machine-learning models at the scale of modern large language models. The design pattern established in the second-generation chip — a systolic matrix-multiply array, on-package HBM, and a tightly integrated host — has been the spine of the TPU program ever since.

The TPU v4, fifth-generation TPU, and sixth-generation TPU generations, announced in 2021, 2023, and 2024 respectively, each progressively increased the chip’s training capability, added support for larger model parallelism, and pushed the chip deeper into Google’s AI training and inference workloads. By 2024 the TPU program was, in scale, comparable to the largest of the merchant AI accelerator programs. The TPU’s role inside Google was, by 2024, no longer niche: the chip was used to train and serve Google’s largest transformer models, including the Gemini family, and it was, on Google’s published disclosures, the principal workhorse of Google’s AI infrastructure. The TPU was, by 2024, also a substantial external business: Google was renting TPU capacity to public-cloud customers at a multi-billion-dollar annual run rate, and the chip was, by virtue of its long-running presence in the public-cloud market, the most visible of the hyperscaler custom-silicon programs.

The seventh-generation TPU, codenamed Ironwood and announced in April 2025, was the first TPU designed specifically for inference rather than training. Ironwood was an explicit recognition by Google that the inference workload had begun to demand its own architectural optimization, and it was the first TPU generation on which Google’s engineering team chose to optimize the inference workload at the expense of the training workload. The chip supported a superpod configuration of up to 9,216 chips, which was, at the time, the largest publicly disclosed superpod configuration of any AI accelerator. The eighth-generation TPU, the TPU v8 announced in April 2026, took the inference-first direction further: Google split the v8 family into two distinct chip designs, the TPU 8t for training and the TPU 8i for inference, with the inference chip designed explicitly for low-latency, memory-bandwidth-heavy serving and the training chip designed for the dense matrix-multiplication workloads of foundation-model training. The Broadcom foundry relationship entered volume production for the TPU 8i in the third quarter of 2026, with mass production targeted for the fourth quarter of 2026.

The strategic significance of the TPU v8 split is substantial. The TPU program is the only hyperscaler custom-silicon program that has, by 2026, run the full design cycle from a single-chip inference accelerator for an internal workload, through a general-purpose training and inference chip for public-cloud customers, through a purpose-built inference chip, and now to a fully bifurcated training-plus-inference chip family. The TPU is also the only hyperscaler custom-silicon program on which the architectural lessons of the past decade have been fully incorporated into the silicon, and it is the only hyperscaler custom-silicon program on which Google has demonstrated that the cost advantage of custom silicon can be substantial enough to support a large external business at meaningful operating margin. Anthropic is, on the public record, raising its TPU deployment from 1 gigawatt in 2026 to 5 gigawatts in 2027, which would make Anthropic the largest single external customer of Google’s custom silicon. The Morgan Stanley research team has estimated that direct TPU sales would grow to 84 billion dollars in 2027 and to 108 billion dollars in 2028, which would make the TPU one of the largest merchant AI chip businesses in the world.

The Microsoft Maia program

Microsoft’s custom-silicon program is, compared to Google’s, much younger and much more strategically focused. Microsoft’s interest in custom AI silicon runs through the early 2017 Project Brainwave experiment, an FPGA-based inference accelerator deployed at modest scale inside Microsoft’s Bing infrastructure. The Project Brainwave program established that Microsoft had the engineering capacity to design and operate custom AI silicon at hyperscale, and it established that Microsoft’s most strategically interesting workload for custom silicon was inference rather than training. The next step was the Maia 100 chip, Microsoft’s first purpose-built AI training accelerator, announced in November 2023 and deployed in production during 2024. The Maia 100 chip was Microsoft’s first entry into the AI training market, and it was, on most measures, a competent but not category-leading chip that was used by Microsoft to train some of its internal AI models and to run some of its AI training workloads alongside Nvidia H100 GPUs.

The strategic shift that produced the Maia 200 chip in 2025 was the recognition by Microsoft’s leadership that the cost of running inference at the scale required by Copilot, GitHub Copilot, and Microsoft 365 Copilot was running into billions of dollars per year and that the only sustainable answer to that pressure was custom silicon optimized for inference. Maia 200, announced in late 2025 and deployed in production in early 2026, was Microsoft’s first chip designed principally for inference. The chip delivered, on Microsoft’s published benchmarks, 1.7 times the tokens per watt of the leading merchant GPU of the previous generation when running Microsoft’s internal AI services. The chip was, by the end of 2026, running a substantial fraction of Microsoft’s production AI inference traffic.

The Maia 300 chip, announced in October 2026, is the third-generation Microsoft AI silicon design and the first that was designed from the ground up for the inference-first architecture. The chip uses the TSMC N2P 2-nanometer process, carries 288 gigabytes of HBM4 memory per accelerator, has 18 terabytes per second of memory bandwidth, and is deployed in 96-chip racks. A 12,288-accelerator superpod is the largest unit of scale. The chip is, on Microsoft’s published performance disclosures, expected to deliver 40 percent more tokens per megawatt than the leading GPU systems on the same inference workloads. Microsoft’s commitment, accompanying the announcement, to deploy Maia at multi-gigawatt scale in 2027 and to make Maia the primary inference system for Microsoft’s own AI products and services, makes Maia 300 the largest single bet on inference-first custom silicon announced by any hyperscaler through the 2026 calendar year.

The distinctive feature of the Microsoft Maia program, in the context of the broader custom-silicon wave, is the closed-cloud decision. Microsoft has, by the end of 2026, not committed to making Maia available to public-cloud customers on a rental basis. The closed-cloud model has substantial strategic advantages for Microsoft: the company captures the full cost advantage of custom silicon inside its own AI services, which gives Microsoft more room to compete with its AI product pricing without sacrificing the margin on the underlying compute. The closed-cloud model also has risks: the program must succeed or fail on the basis of Microsoft’s internal AI services alone, and the amortization of the silicon investment is, accordingly, smaller than it would be if the silicon were rented externally. The closed-cloud decision is, however, an explicit strategic choice, and the October 2026 announcement is the moment at which Microsoft fully committed to it.

Amazon’s Trainium and Inferentia programs

Amazon’s custom-silicon program is, after Google’s, the most diverse and the most externally visible of the hyperscaler programs. Amazon’s first purpose-built AI accelerator was Inferentia, a 2018-era inference chip designed for the AWS inference workloads that power Alexa and the AWS machine-learning services. Inferentia was deployed in 2019 and has, on Amazon’s published disclosures, been deployed in ever-larger volumes since. The second-generation Inferentia chip, Inferentia 2, was announced in 2022 and has, on the public record, been the principal AI inference accelerator for AWS’s public-cloud customers in the late 2020s. Trainium, Amazon’s training-focused accelerator, was first announced in 2020 and entered production in 2021. Trainium 2, the second-generation training accelerator, was announced in 2022 and entered production in 2023. The Trainium and Inferentia families were, in design philosophy, deliberately kept separate: Trainium optimized for training, Inferentia for inference, with each family maintained on its own design trajectory.

The third-generation Trainium chip, Trainium 3, was announced in 2025 and entered production in 2026. Trainium 3 is the chip on which most of Amazon’s reported AI training business now runs, and it is the chip that has, on Amazon’s earnings disclosures, turned the Trainium business into a 25-billion-dollar annual revenue run rate. The 25-billion-dollar number is the largest disclosed revenue figure associated with any hyperscaler custom-silicon program, and it is the figure that has established Trainium as the most commercially successful of the hyperscaler custom-silicon programs in dollar terms. Anthropic, the AI lab behind the Claude family of models, is Trainium’s largest disclosed external customer; Anthropic has, on the public record, committed to running a substantial fraction of its training and inference workloads on Trainium silicon through the late 2020s.

The Trainium and Inferentia programs together account for the largest deployed base of hyperscaler custom AI silicon in the world. The most recent AWS disclosures indicate that the Trainium 3 chip is being shipped to public-cloud customers, to AI-lab customers including Anthropic, and to AWS’s own internal AI services at a combined annual volume that is comparable in scale to the largest merchant AI accelerator businesses. The Inferentia program continues to serve the AWS inference workloads that underpin the AWS AI services, and the combined Trainium-and-Inferentia product family is, on Amazon’s earnings disclosures, one of the largest contributors to AWS’s operating margin in the AI era. Amazon’s strategic position, in 2026, is the most financially successful of the hyperscaler custom-silicon positions and the most exposed to the risk that an architectural shift in the AI workload, away from the training-and-inference split that Trainium and Inferentia are designed to support, would force a re-design of the silicon.

The architectural lesson from the Trainium success is that custom silicon pays off most clearly when the custom silicon has a clear and stable workload. Trainium’s success has come from a stable workload definition — the training of large transformer models — that has allowed Amazon to design, produce, and improve the chip over three generations without being forced to redesign the silicon to chase a moving target. Inferentia’s success has been more circumscribed because the inference workload has been less stable than the training workload; the architectural shifts toward long-context windows, multi-modal models, and agentic workloads that have characterized the late 2020s have forced Inferentia to upgrade more rapidly than Trainium has needed to. The architectural lesson is also visible in OpenAI’s Jalapeño program: the design choices in Jalapeño are, on the public record, heavily informed by the OpenAI inference workload rather than by the general-purpose AI training workload, and the chip is expected to be substantially optimized for the OpenAI workload rather than for the broader market.

The OpenAI, Meta, Anthropic, and Qualcomm-Amazon programs

Beyond the three largest hyperscaler programs, a substantial number of additional custom-silicon programs are running. The most strategically interesting is OpenAI’s Jalapeño chip, designed and brought to tape-out in nine months using OpenAI’s own models and announced in mid-2026 in collaboration with Broadcom and manufactured by TSMC. The Jalapeño chip is a first-generation inference accelerator specifically designed for the OpenAI workload, and OpenAI has reported, as of August 2026, that the first-generation chip is outperforming Nvidia hardware on certain workloads. The Jalapeño program is one of several OpenAI moves toward vertical integration: the company has, in parallel, entered into a chip-development relationship with Samsung, has signed a custom-chip agreement with Broadcom for the next generations of Jalapeño, and has signed substantial multi-year custom-silicon commitments with TSMC. The strategic implication is that OpenAI is no longer willing to depend on the merchant GPU market for the silicon on which its models run, and that OpenAI is, in effect, becoming the third major Western AI lab after Google and Amazon to operate a substantial custom-silicon program.

Meta’s Iris chip is the next-largest of the non-hyperscaler programs. Meta announced Iris as part of its Meta Training and Inference Accelerator program in late 2024 and entered volume production in September 2026, with the chip fabricated by TSMC and the design partnership with Broadcom. Iris is, on the public record, a general-purpose AI accelerator that is being deployed at scale inside Meta’s training and inference fleets, and Meta’s published roadmap calls for a new chip every six months through 2027. The cadence of the Iris roadmap is a substantial departure from the conventional hyperscaler cadence of a chip every two to three years, and it is one of the more aggressive custom-silicon roadmaps publicly disclosed by a Western AI company.

Anthropic’s chip program is, at the time of writing, the least publicly disclosed of the major non-hyperscaler programs. Anthropic has confirmed that it is exploring its own chip designs even as it continues to use chips from Nvidia, Amazon, and other suppliers, and the company is reported to have already hired substantial chip-design talent from outside the company. The strategic motivation for an Anthropic chip program is, in the public record, the same as for Microsoft’s: the company is the largest single customer of Anthropic inference, and the cost of running that inference is the largest single component of Anthropic’s operating cost. Anthropic’s chip strategy is, accordingly, likely to be substantially customized to the Claude model workload.

The Qualcomm-Amazon agreement, announced in September 2026, is the most architecturally distinctive of the recent non-hyperscaler programs. The agreement includes a potential purchase of up to 60 billion dollars in AI data-centre chips and related products over the multi-year horizon, plus work on optical connectivity and on custom silicon. The partnership is interesting because it brings Qualcomm, the smartphone chip company, into the data-centre custom-silicon market at a substantial scale, and because it incorporates optical connectivity into the partnership’s technology scope. The partnership is, on the financial scale, large enough to be material for Qualcomm and small enough to be incremental for Amazon. Its strategic significance is in its precedent: it is the first substantial partnership of a hyperscaler with a merchant chip company on a custom-silicon deal, and it suggests that the hyperscaler custom-silicon wave is going to be complemented, in the late 2020s, by substantial merchant-chip partnerships with hyperscalers that prefer to buy custom silicon from outside rather than to design it fully in-house.

The cumulative scale of the custom-silicon wave, by late 2026, is substantial. The market-research firms Makreo Research and Yole Group have estimated that the share of data-centre chip value held by custom in-house silicon is set to rise from approximately 16.6 percent in 2025 to approximately 30 percent by 2035, with the absolute revenue of custom in-house silicon growing faster than the merchant GPU market over the same period. The AI chip market as a whole is projected to reach approximately 1.5 trillion dollars in revenue by 2031, with custom in-house silicon accounting for an increasingly large fraction of the total. The custom-silicon wave is, accordingly, not a passing fashion; it is the principal driver of the AI chip market’s growth over the rest of the decade, and it is the development that is going to shape the structure of the AI infrastructure industry through the 2030s.

What the wave means for Nvidia and for the merchant GPU market

Nvidia’s response to the custom-silicon wave has been, on the public record, two-pronged. The first prong has been the aggressive scaling of the merchant GPU product line: the Vera Rubin platform, announced in early 2026, is the first full rack-scale system from Nvidia and includes the Vera CPU, the Rubin GPU, the NVLink 6 switch, the ConnectX-9 SuperNIC, the BlueField-4 DPU, and the Spectrum-6 Ethernet switch. The platform is, on Nvidia’s published performance disclosures, a substantial improvement over the previous-generation Grace Blackwell system on agentic-AI workloads (up to 30x better, with up to 60x at some points on the Pareto frontier). The platform’s purpose is to keep the merchant GPU at the centre of the largest-scale AI training and inference deployments, and to make the merchant GPU so substantially better than the custom alternatives that even a custom-silicon-mature hyperscaler continues to use substantial amounts of merchant GPU capacity for the most demanding workloads.

The second prong has been the acquisition of Groq at the end of 2025 and the integration of the Groq 3 language-processing unit into the Vera Rubin platform. The Groq 3 LPU is, like Microsoft’s Maia 300 and Google’s TPU 8i, a memory-centric inference accelerator with a substantial amount of on-die SRAM connected directly to the floating-point units. The acquisition gives Nvidia a second architectural point of presence in the inference market and allows the company to offer customers a hybrid GPU-plus-LPU inference rack in which the GPU handles the prefill workload and the LPU handles the decode workload. The hybrid approach is a defensive response to the architectural divergence that the custom-silicon wave is causing: by owning both architectural points, Nvidia can offer customers a single-vendor inference system that is competitive with the bifurcated custom-silicon systems on the workload mix that the customer actually has, while keeping the GPU at the centre of the design. The Groq 3 acquisition was the largest AI infrastructure M&A; deal of the 2025 calendar year, and it is one of the clearest indications yet of how seriously Nvidia is taking the inference-first architectural shift.

The strategic implication for Nvidia is that the merchant GPU market is going to be a smaller fraction of the AI infrastructure market than it was in the early 2020s, but that it is going to remain the largest single chip-vendor position in absolute dollar terms. Microsoft’s Maia, Google’s TPU, Amazon’s Trainium, Meta’s Iris, OpenAI’s Jalapeño, and the long tail of inference-specialized accelerators are all taking share away from the merchant GPU, but they are doing so at the same time as the overall AI chip market is growing fast enough that Nvidia’s absolute revenue from the merchant GPU is still growing. The Bain research team’s framing of the late-2020s AI infrastructure industry as one in which custom silicon is “the fastest-growing segment of data-center compute chips” is correct as a description of relative growth; the absolute growth of the merchant GPU is, however, still substantial. Nvidia’s market position is, accordingly, not collapsing. It is, however, becoming a smaller fraction of a much larger whole, and the corporate strategy Nvidia is executing in response is to be present in every architectural direction in which the AI infrastructure industry is moving.

The custom-silicon wave is, accordingly, the defining structural feature of the late-2020s AI infrastructure industry. The hyperscalers are not retreating from the custom-silicon bet; they are accelerating it, with each generation of custom silicon delivering larger absolute performance and cost advantages over the previous generation. The merchant GPU market is responding by diversifying, both architecturally (with the acquisition of Groq) and at the system level (with the Vera Rubin rack-scale platform). The collective effect of these two trends is an AI infrastructure industry that is, by the late 2020s, substantially more architecturally diverse than the AI infrastructure industry of the early 2020s, and that has, in the process, become substantially more economically efficient at serving the workloads on which the AI service economy depends. The custom-silicon wave is good for the AI service economy. It is also a substantial long-term challenge for Nvidia. The next several years will determine how that long-term challenge resolves.