The HBM4 Era and the Memory Wall

By Nekud October 16, 2026 17 min read

The phrase “memory wall” has, by late 2026, become one of the most-cited pieces of jargon in the AI infrastructure industry, just as the phrase “power wall” has. The memory wall is the moment at which the memory system of an AI compute fleet — the high-bandwidth memory on the accelerators, the DRAM on the server boards, the SSDs and the storage-class memory behind them — fails to keep up with the floating-point throughput of the AI compute silicon, with the result that the AI compute silicon sits idle waiting for memory to supply the data that the silicon needs to do useful work. The memory wall is, in the AI infrastructure industry, a particularly acute problem because the AI compute workload is one of the most memory-intensive workloads in modern computing. The training of a transformer model is memory-bound as well as compute-bound, the inference of a transformer model is memory-bound more than it is compute-bound, and the agentic workload that is becoming the dominant production workload is memory-bound to a degree that the earlier generation of AI compute silicon was not designed for.

The HBM4 generation of high-bandwidth memory is the industry’s response to the memory wall. HBM4 is the fourth generation of the HBM standard that the JEDEC standards body has maintained since the mid-2010s, and it is the first generation designed specifically for the AI compute workload rather than for the broader high-performance computing workload that the earlier generations of HBM were designed for. HBM4 is now in production for the Nvidia Vera Rubin platform; it is the foundation of the Microsoft Maia 300 inference accelerator’s 288-gigabyte and 18-terabyte-per-second memory subsystem; it is the memory subsystem for the Google TPU v8 family and for the OpenAI Jalapeño chip; and it is the memory subsystem for the AMD MI series and for the long tail of AI accelerator products that the late 2020s are producing. The HBM4 generation is, in 2026, the largest single memory product in the semiconductor industry by revenue, and it is the largest single binding-constraint relief in the AI compute fleet’s recent architectural history.

The strategic significance of HBM4 is, in the late 2020s, larger than the strategic significance of any single floating-point accelerator. The reason is that the memory subsystem is, by the published industry data, the single largest share of the AI server bill-of-materials. The Makreo Research report’s estimate that semiconductors account for more than 95 percent of the content value of a leading AI server rack, and that the memory subsystem accounts for a substantial fraction of that 95 percent, makes the memory subsystem the single largest piece of the AI compute cost structure. The HBM4 generation is, accordingly, the single largest determinant of the AI compute industry economics for the rest of the decade, and the supplier of the HBM4 memory is the single most strategically important company in the memory industry since the emergence of the AI compute market.

This essay traces the HBM4 generation from its standards development in the early 2020s through its production ramp in 2026, the three principal suppliers (Samsung, SK Hynix, and Micron), the architectural choices that distinguish HBM4 from the earlier HBM3 and HBM3e generations, the principal HBM4 customers, and the strategic implications for the AI compute industry. It focuses on three threads. The first is the engineering thread: what HBM4 is, how it differs from its predecessors, and what architectural choices the AI compute silicon is making to exploit it. The second is the supplier thread: the strategic positioning of Samsung, SK Hynix, and Micron in the HBM4 market, the supply constraints on the HBM4 generation, and the implications for AI compute customers. The third is the strategic thread: the long-term trajectory of the memory wall, the role of the memory subsystem in the AI compute stack, and the implications for the AI compute industry’s evolution over the rest of the decade. The thesis the essay develops is the following. The HBM4 generation is the single largest relief of the memory-wall constraint of the AI compute era, and the HBM4 market is the single most strategically consequential market in the semiconductor industry in the late 2020s.

What HBM4 is and how it differs from its predecessors

The HBM4 generation is the fourth generation of the HBM standard, and the differences between HBM4 and HBM3e are substantial. The HBM3e generation, which had been in volume production for approximately two years by late 2026, was the first HBM generation designed primarily for the AI compute workload. HBM3e delivered, on the standard published JEDEC specification, between 800 and 1,000 gigabytes per second of bandwidth per stack, with a stack capacity of between 16 and 32 gigabytes. HBM4 delivers substantially more bandwidth and substantially more capacity per stack: between 1,200 and 1,500 gigabytes per second of bandwidth per stack, with a stack capacity of between 24 and 48 gigabytes. The combined improvement, on a per-stack basis, is on the order of 50 percent more bandwidth and 50 percent more capacity than HBM3e. The HBM4 generation is also the first HBM generation designed for the 16-high stacked-die configuration that allows the largest possible stack capacity per footprint, with HBM3e limited to 8-high configurations in the most common deployments.

The architectural improvements that produced the HBM4 generation’s larger bandwidth are partly in the silicon and partly in the packaging. On the silicon side, the HBM4 die is manufactured on a 1b-nanometer DRAM process (compared with the 1c-nanometer process used for the most advanced HBM3e die) and incorporates several generations of circuit-design improvement that have allowed the per-pin data rate to climb from 8 gigabits per second in HBM3e to roughly 12 gigabits per second in HBM4. On the packaging side, the HBM4 stack uses a more advanced through-silicon-via technology and a more advanced chip-on-wafer-on-substrate (CoWoS) heterogeneous-integration packaging flow that has allowed the number of stacks per accelerator to climb from the 4 to 6 stacks common in HBM3e deployments to the 6 to 8 stacks that are becoming common in HBM4 deployments. The combined silicon-and-packaging improvements have pushed the per-accelerator memory bandwidth and the per-accelerator memory capacity substantially above what HBM3e could deliver.

The HBM4 generation is also the first HBM generation designed for the agentic AI workload. The earlier HBM generations were designed for the AI training workload, in which the memory pattern is dominated by the random-access pattern of the training gradient and by the all-reduce traffic of large-model training. The HBM4 generation is designed for the AI inference workload as well as for the training workload, and it incorporates several architectural choices that are specific to the inference workload. The most consequential of these is the larger stack capacity: the larger stack capacity allows a single HBM4-equipped accelerator to hold a substantially larger fraction of a frontier model’s weights on-package, which reduces the off-package memory access during decode and improves the decode latency. The larger stack capacity also allows the AI compute customer to choose a smaller number of accelerators per model serving, which improves the efficiency of the serving stack as a whole.

The HBM4 generation is, in addition, the first HBM generation in which the standards body has explicitly accommodated the heterogeneous-integration packaging choices that the AI compute customers are making. The JEDEC HBM4 specification includes, for the first time, explicit provisions for the use of HBM4 with the most advanced 2.5D and 3D packaging technologies (including CoWoS and the equivalent packaging flows at the competing foundries). The explicit accommodation of the packaging choices is, in substance, a recognition that the memory subsystem of an AI compute accelerator is no longer a separately-packaged component but a co-designed component of the overall AI compute package. The HBM4 generation’s co-design with the packaging is one of the things that distinguishes it from the earlier generations, and it is one of the reasons why the HBM4 generation’s architectural improvements are larger than the formal JEDEC specification numbers would suggest.

The three HBM4 suppliers and the supply-side dynamic

The HBM4 generation is supplied by three companies: Samsung, SK Hynix, and Micron. The three companies have been the principal HBM suppliers for every generation of HBM since HBM2, and they are the only three companies with the technology base and the production capacity to produce HBM at the scale and the quality that the AI compute industry requires. The relative positions of the three companies in the HBM4 market are, however, substantially different from their positions in the HBM3e market, and the shifts in market position are one of the principal strategic dynamics in the HBM market in 2026.

SK Hynix has, by the HBM4 generation, established itself as the largest HBM4 supplier and the principal supplier to the largest AI compute customers. SK Hynix’s published disclosure through the third quarter of 2026 indicates that the company has been the dominant supplier of HBM4 to Nvidia’s Vera Rubin platform, that it has substantial supply commitments to Microsoft for the Maia 300 platform, and that it has a leading share of the AMD HBM4 supply business. The dominance reflects SK Hynix’s substantial first-mover advantage in the HBM4 generation, with the company having been the first of the three HBM suppliers to bring an HBM4 product to volume production, and the first to qualify the HBM4 product with the largest AI compute customers. SK Hynix’s HBM4 revenue is, on the company’s published guidance, on track to exceed 30 percent of the company’s total revenue in 2026, and the HBM4-driven revenue growth is the single largest contributor to the company’s growth in the year.

Samsung has, by the HBM4 generation, made substantial progress in its HBM4 product and is, on the company’s published disclosure, the second-largest HBM4 supplier with substantial supply commitments to several AI compute customers including, on the public record, OpenAI. Samsung’s HBM4 product line is, on the engineering detail, broadly comparable to SK Hynix’s in terms of bandwidth and capacity per stack, but Samsung has historically been slower to qualify with the largest AI compute customers, with the qualification cycle being the single largest gating factor on Samsung’s HBM4 revenue. The September 2026 OpenAI-Samsung partnership announcement is, in substance, a way for Samsung to substantially accelerate its qualification cycle with a major AI compute customer, and the partnership reflects Samsung’s strategic intention to take a larger share of the HBM4 market through the late 2020s.

Micron is, by the HBM4 generation, the third-largest HBM supplier with a substantial but smaller share of the HBM4 market. Micron’s HBM4 product line is, on the engineering detail, broadly comparable to those of SK Hynix and Samsung, but Micron has, on the company’s published disclosure, been the slowest of the three suppliers to bring an HBM4 product to volume production. Micron’s strategic positioning in the HBM4 generation is, in 2026, focused on the long-term catch-up path rather than on the immediate market-share competition. The company is, on the basis of its published technology roadmap, on track to introduce its second-generation HBM4 product, the HBM4e product line, with substantial bandwidth improvements over the first-generation HBM4 product. The HBM4e product line is intended to allow Micron to take a larger share of the HBM4 market through the late 2020s, with the share gain being the principal strategic objective of the HBM4e product introduction.

The supply-side dynamic of the HBM4 generation is, in 2026, one of substantial tightness. The demand for HBM4 from the AI compute customers has substantially outrun the supply, and the HBM4 supply constraints have been one of the principal binding constraints on the AI compute buildout. The supply tightness has been most acute for the HBM4 products with the largest stack capacities and the highest bandwidths, with several of the AI compute customers reporting, in 2026, that their accelerator deployments have been delayed by HBM4 supply constraints rather than by accelerator supply constraints. The Makreo Research report’s estimate that the AI compute semiconductor revenue will exceed 1.2 trillion dollars by 2028 is, in large part, a reflection of the HBM4 supply tightness: the price of HBM4 per stack has been substantially above the price of HBM3e per stack, and the AI compute industry’s bill-of-materials reflects the higher HBM4 price.

How HBM4 is changing the architecture of AI silicon

The HBM4 generation is not just an incremental improvement over HBM3e; it is enabling substantial architectural changes in the AI compute silicon. The most consequential of these changes is the move toward larger on-package memory totals on the AI compute accelerators. The Microsoft Maia 300 chip, with its 288 gigabytes of HBM4 memory, holds, on a per-accelerator basis, roughly twice the memory of the previous-generation Maia 200 chip. The 288-gigabyte memory total is enough to hold a substantial fraction of a frontier model’s weights on-package, which substantially reduces the off-package memory access during decode and which substantially improves the decode latency. The implication is that the Maia 300 chip is, in performance terms, substantially faster on the decode-heavy workloads that dominate Microsoft’s AI services than the Maia 200 chip would be, even if the floating-point throughput of the two chips were comparable.

The second architectural change enabled by HBM4 is the move toward higher stack counts per accelerator. The earlier generations of AI compute accelerators typically had 4 to 6 HBM stacks; the HBM4 generation is allowing accelerators to have 6 to 8 HBM stacks, and the larger stack counts are allowing the AI compute customers to push the per-accelerator memory totals into the 256 to 512 gigabyte range that is becoming the standard for the largest AI compute accelerators. The larger per-accelerator memory totals are, in turn, allowing the AI compute silicon to do more of the AI compute workload from on-package memory, with correspondingly less off-package memory access and correspondingly less dependence on the off-package memory system. The architecture is, in substance, increasingly “memory-saturated” with the AI compute silicon designed to be limited by the AI model’s weights rather than by the memory subsystem’s capacity.

The third architectural change enabled by HBM4 is the move toward higher per-stack bandwidth and the corresponding increase in the per-accelerator memory bandwidth. The Maia 300 chip’s 18 terabytes per second of memory bandwidth is, on a per-accelerator basis, roughly 50 percent higher than the previous-generation Maia 200 chip’s memory bandwidth. The higher memory bandwidth is, in turn, allowing the AI compute silicon to do more floating-point work per byte of memory accessed, which improves the floating-point throughput efficiency of the AI compute silicon. The architectural point is that HBM4 allows the AI compute silicon to be more thoroughly memory-fed than the earlier generations of AI compute silicon, and the more-thorough memory-feeding is, in turn, what allows the floating-point throughput to scale without being constrained by the memory subsystem.

The fourth architectural change enabled by HBM4 is the move toward more sophisticated memory-system architectures. The earlier generations of AI compute accelerators had, in most cases, a single HBM subsystem that served all of the floating-point units on the chip. The HBM4 generation is allowing the AI compute silicon designers to build multiple HBM subsystems, with each subsystem serving a different part of the floating-point unit array. The multiple-subsystem architecture is, in turn, allowing the AI compute silicon to exploit the higher HBM4 bandwidth more thoroughly than the single-subsystem architecture could. The multiple-subsystem architecture is one of the principal architectural moves that the AI compute silicon designers have made to exploit the HBM4 generation’s improvements.

The server-DRAM and storage complement to HBM4

The HBM4 generation is the most strategically consequential memory subsystem in the AI compute stack, but it is not the only memory subsystem that matters. The server DRAM and the storage subsystems are also important parts of the AI compute memory stack, and the late 2020s are seeing substantial changes in those subsystems as well. The server DRAM market is, in 2026, in substantial supply tightness, with the demand from the AI compute customers substantially outrunning the supply. The Gartner expectation that semiconductor revenue will grow 92 percent in 2026 to 1.56 trillion dollars is, in large part, a reflection of the server DRAM price increases that the AI compute demand has driven. The server DRAM supply tightness has, in turn, driven substantial new investment in DRAM fab capacity at Samsung, SK Hynix, and Micron, with the new capacity expected to come online in 2027 and 2028.

The storage subsystem has also been substantially affected by the AI compute demand. The AI compute workload is, in the late 2020s, one of the largest single consumers of high-performance solid-state storage, with the storage subsystem used for the model checkpointing, the training data, and the inference-serving environment. The demand from AI compute customers has driven substantial new investment in high-performance SSD technology, with the largest SSD suppliers — Samsung, Kioxia, Western Digital, and Micron — all reporting that AI compute demand is the principal driver of their high-end SSD product revenue. The storage subsystem is, in revenue terms, a smaller market than the HBM4 market, but it is a strategically important complement to the HBM4 market and is one of the principal sources of the bill-of-materials cost of an AI compute rack.

The server DRAM and the storage subsystems are also the memory subsystems in which the AI compute customers are doing the most aggressive architectural experimentation. The HBM4 generation has, by late 2026, substantially converged on a single architectural point, with all of the major AI compute accelerators using comparable HBM4 stacks in comparable configurations. The server DRAM and the storage subsystems have, in contrast, a much wider range of architectural choices, with the AI compute customers experimenting with different types of server DRAM (including the new MRAM and the new PCM technologies that several memory suppliers are developing for the AI compute market), different types of high-performance storage (including the new computational storage devices that incorporate processing capability into the storage device itself), and different types of memory-network connectivity (including the new chip-to-chip connectivity technologies that allow the AI compute accelerator to share memory with the storage subsystem in ways that the earlier generations of AI compute silicon could not).

The strategic implications of the HBM4 era

The HBM4 era has strategic implications for the AI compute industry that extend well beyond the immediate generation of HBM4 products. The first strategic implication is that the memory subsystem has become the largest single category of the AI compute industry’s bill-of-materials, and the suppliers of the memory subsystem have, accordingly, become some of the largest and most strategically positioned companies in the semiconductor industry. SK Hynix, in particular, has moved from being a memory supplier with substantial but not dominant market position in 2020 to being the principal supplier of the largest single category of AI compute silicon in 2026, with the company’s revenue and stock price having substantially outperformed the broader semiconductor index over the same period. The shift in SK Hynix’s position is the most consequential single corporate-strategy outcome of the AI compute era in the memory industry.

The second strategic implication is that the supply tightness of HBM4 has become one of the principal binding constraints on the AI compute buildout. The AI compute customers have, in 2025 and 2026, substantially expanded their multi-year supply commitments with the HBM4 suppliers, with several AI compute customers signing supply commitments that stretch through 2030. The supply commitments are, in dollar terms, among the largest single supply commitments in the history of the semiconductor industry, and they are reshaping the customer-supplier relationships in the memory industry. The supply commitments are also, in turn, locking in the HBM4 generation’s pricing through the period of the supply commitments, with the implication that the HBM4 price is going to remain elevated relative to the HBM3e price for substantially longer than the historical pattern would suggest.

The third strategic implication is that the HBM4 generation has substantially raised the bar for what counts as a competitive AI compute accelerator. The earlier generations of AI compute accelerators could, in some cases, be designed without substantial HBM sophistication; the HBM4 generation is, in contrast, a generation in which competitive AI compute silicon has to be designed around the HBM4 architectural constraints. The implication is that the AI compute accelerator market of the late 2020s is going to be more difficult to enter than the AI compute accelerator market of the early 2020s, with the consequence that the incumbent AI compute accelerator vendors are going to face less new-entrant competition than they did in the earlier years.

The fourth strategic implication is that the HBM4 generation has accelerated the architectural shift toward memory-centric inference silicon discussed earlier in this essay series. The larger per-stack capacity of HBM4, the higher stack counts per accelerator, and the higher per-accelerator memory bandwidth are all of the architectural improvements that benefit the memory-centric inference design more than they benefit the floating-point-centric training design. The implication is that the HBM4 generation is, in addition to its substantial per-generation improvements in the HBM subsystem, also accelerating the architectural shift that the AI compute industry is going through.

The HBM4 era is, in summary, the largest single change in the AI compute memory subsystem since the emergence of the AI compute market. The HBM4 generation is enabling substantial architectural changes in the AI compute silicon, is the single largest relief of the memory-wall constraint, and is the principal determinant of the AI compute industry’s economics for the rest of the decade. The HBM4 market is, accordingly, the most strategically consequential market in the semiconductor industry in the late 2020s, and the suppliers of HBM4 are the most strategically important companies in the memory industry. The trajectory of the HBM4 generation is, in many respects, the trajectory of the AI compute industry itself.