The Chip to Grid Data Center

By Nekud October 16, 2026 16 min read

The phrase “chip-to-grid” emerged from the Open Compute Project 2026 summit as the organizing principle for the next generation of AI data-centre design. The phrase describes what the OCP’s 2026 reporting described as a recognition that the AI data centre of the late 2020s is no longer a building that hosts servers; it is a fully engineered system that runs from the silicon at the bottom through the rack and the cooling and the power distribution at the bottom, to the network and the storage and the workload scheduling in the middle, to the grid connection and the power market participation at the top. The shift is being driven, on the OCP’s reporting, by the agentic AI workload, which has forced data-centre operators to design every layer of the stack together rather than designing each layer in isolation. The result is that the AI data centre is becoming a fully integrated chip-to-grid system that is engineered end-to-end for the AI workload, rather than a building that hosts AI racks that have been retrofitted into a conventional data-centre architecture.

The chip-to-grid design philosophy has four architectural components. The first is the rack as the unit of scale. The conventional data centre of the early 2020s was designed around the individual server, with the rack as a relatively low-density mechanical frame. The AI data centre of the late 2020s is designed around the rack as the unit of scale, with the rack being a fully engineered system that integrates the AI compute silicon, the memory subsystem, the network interfaces, the cooling system, and the power-delivery system. Microsoft’s Maia 300 rack, with 96 accelerators in 24 four-accelerator trays, and the AMD Helion specification, with up to 32,768 CPU cores per rack, are both examples of the rack-as-unit-of-scale architecture. Nvidia’s NVL576, AMD’s Helios, AWS’s NL72, and Google’s Ironwood are all examples of the same architecture, with the rack being the smallest unit at which an AI workload can be deployed.

The second component is the superpod as the unit of aggregation. The conventional data centre of the early 2020s was designed around the individual data-centre building, with the rack as the unit of scale within the building. The AI data centre of the late 2020s is designed around the superpod as the unit of aggregation, with the superpod being a fully engineered system that integrates hundreds or thousands of racks into a single AI workload deployment. Microsoft’s Maia 300 superpod, with 12,288 accelerators in 128 racks, and Google’s TPU 8i superpod, with 9,216 chips, are both examples of the superpod-as-unit-of-aggregation architecture. The superpod is, in turn, the smallest unit at which a single AI workload can be deployed at the scale that the AI workload requires.

The third component is the data-centre campus as the unit of infrastructure. The conventional data centre of the early 2020s was designed around the individual data-centre building, with the campus being a relatively loose collection of buildings. The AI data centre of the late 2020s is designed around the campus as the unit of infrastructure, with the campus being a fully engineered system that integrates the data-centre buildings, the power-generation capacity, the cooling capacity, the network connectivity, and the security infrastructure. The AI data-centre campuses being built by Microsoft, Google, Amazon, Meta, and the neoclouds are all designed around the campus-as-unit-of-infrastructure architecture, with the campus being the smallest unit at which a substantial AI workload can be deployed at the scale that the AI workload requires.

The fourth component is the grid connection as the unit of energy. The conventional data centre of the early 2020s drew power from the grid and treated the grid as a passive source. The AI data centre of the late 2020s is designed around the grid connection as the unit of energy, with the data centre actively participating in the grid’s demand-response markets, the grid’s frequency regulation, and the grid’s capacity markets. The data centre is, in this design, not just a power consumer; it is a grid asset, and the value of the data-centre load is determined, in part, by the grid services that the data centre can provide.

This essay traces the chip-to-grid design philosophy through four threads. The first is the silicon thread: how the AI compute silicon is being designed for the rack-as-unit-of-scale architecture and for the superpod-as-unit-of-aggregation architecture. The second is the rack thread: how the AI data-centre rack is being designed for the rack-as-unit-of-scale architecture and for the chip-to-grid system as a whole. The third is the data-centre thread: how the AI data-centre campus is being designed for the campus-as-unit-of-infrastructure architecture and for the chip-to-grid system as a whole. The fourth is the grid thread: how the AI data-centre load is being designed for the grid-as-unit-of-energy architecture and for the chip-to-grid system as a whole. The thesis the essay develops is the following. The chip-to-grid design philosophy is the most consequential change in the data-centre industry’s design principles since the emergence of the internet data centre, and the philosophy is going to be the principal organizing principle for the AI data-centre industry through the rest of the decade.

The silicon layer of the chip-to-grid system

The silicon layer of the chip-to-grid system is the layer at which the AI compute silicon is being redesigned to fit the chip-to-grid design philosophy. The AI compute silicon of the early 2020s was designed for the individual-server architecture: each AI compute accelerator was a self-contained unit that connected to a host CPU through a PCIe interface and that received its data and its instructions from the host CPU. The AI compute silicon of the late 2020s is being redesigned for the rack-as-unit-of-scale architecture: each AI compute accelerator is being designed to be a fully integrated component of the rack, with the accelerator’s interfaces to the network, to the memory subsystem, to the power delivery, and to the cooling system all designed for the rack as the unit of scale rather than for the individual accelerator as the unit of scale.

The most visible architectural change in the silicon layer is the change in the network interface. The AI compute accelerator of the early 2020s had a relatively modest network interface, with a single network connection per accelerator that connected the accelerator to the host CPU and to the broader network through the host CPU. The AI compute accelerator of the late 2020s is being designed with substantial dedicated network bandwidth, with the accelerator having multiple network connections that go directly to the rack-level network rather than through the host CPU. The dedicated network bandwidth allows the accelerator to participate in the rack-level network as a first-class citizen, with the accelerator able to communicate directly with the other accelerators in the rack and to participate in the rack-level collective operations. Microsoft’s Maia 300 accelerator, with its built-in scale-up Ethernet network interface, is an example of the dedicated-network-bandwidth architecture.

The second architectural change in the silicon layer is the change in the memory subsystem. The AI compute accelerator of the early 2020s had a relatively modest on-package memory subsystem, with the accelerator’s HBM memory sized to fit the typical AI training workload of the period. The AI compute accelerator of the late 2020s is being designed with a substantially larger on-package memory subsystem, with the accelerator’s HBM memory sized to hold a substantial fraction of a frontier model’s weights. The larger on-package memory subsystem is, on Microsoft’s published performance disclosures for Maia 300, the single largest determinant of the chip’s inference performance: the 288 gigabytes of HBM4 per accelerator is enough to hold a substantial fraction of a frontier model’s weights on-package, which substantially reduces the off-package memory access during decode.

The third architectural change in the silicon layer is the change in the power-delivery interface. The AI compute accelerator of the early 2020s had a relatively modest power-delivery interface, with the accelerator’s power delivered through a conventional PCIe power connector or a conventional server-board power connector. The AI compute accelerator of the late 2020s is being designed with a substantially more capable power-delivery interface, with the accelerator’s power delivered through a dedicated high-current power connector that is part of the rack’s power-delivery architecture. The dedicated power connector allows the rack to deliver the high current that the latest AI compute accelerators require, and the connector’s specifications are designed for the rack-level power-delivery architecture rather than for the individual accelerator.

The fourth architectural change in the silicon layer is the change in the cooling interface. The AI compute accelerator of the early 2020s had a relatively modest cooling interface, with the accelerator’s cooling handled by a conventional heat sink and a conventional air-cooling airflow. The AI compute accelerator of the late 2020s is being designed with a substantially more capable cooling interface, with the accelerator’s cooling handled by a cold plate that is part of a rack-level liquid-cooling loop. The cold plate is, in effect, an integral part of the rack’s cooling architecture, and the rack’s cooling architecture is designed around the cold plate as the unit of cooling rather than around the individual accelerator’s heat sink.

The rack layer of the chip-to-grid system

The rack layer of the chip-to-grid system is the layer at which the AI compute silicon, the memory subsystem, the network, the cooling system, and the power-delivery system are integrated into a single engineered unit. The rack layer has been, in late 2026, the principal architectural change in the data-centre industry, and the rack layer’s design is being standardized around the rack-as-unit-of-scale architecture. The Rack-Scale Architecture specification that AMD introduced at OCP 2026 (the Helion specification) is the most visible expression of the rack-as-unit-of-scale architecture. The Helion specification describes the rack as a fully engineered unit, with the rack-level interconnect, the rack-level cooling, and the rack-level power delivery all designed together rather than being assembled from conventional server components.

The Intel Crescent Island GPU D-Series rack is another example of the rack-as-unit-of-scale architecture. The GPU D-Series rack is designed to host up to eight Intel GPU D-Series accelerators in an air-cooled rack configuration, with the rack’s power, cooling, and network interfaces all designed for the rack-as-unit-of-scale. The Crescent Island GPU D-Series accelerator has 1.3 peak PFLOPS of FP8 compute and 1.5 TB/s of memory bandwidth, and the rack-level design supports up to 480 GB of LPDDR5X memory per accelerator. The TDP of the rack is 350 watts per accelerator, and the rack is designed for inference workloads. The Intel GPU D-Series rack is, in essence, an example of the rack-as-unit-of-scale architecture for an inference-optimized AI compute workload.

The NVIDIA NVL576 rack is a third example of the rack-as-unit-of-scale architecture. The NVL576 rack contains 72 of NVIDIA’s Vera Rubin GPU in a rack-scale configuration, with the rack’s NVLink 6 network, ConnectX-9 SuperNIC, BlueField-4 DPU, and Spectrum-6 Ethernet switch all designed for the rack-as-unit-of-scale. The NVL576 rack is, on NVIDIA’s published performance disclosures, the principal unit at which a single AI workload is deployed in the Vera Rubin platform, and the rack’s design is the basis for the larger superpod designs.

The Microsoft Maia 300 rack, the AWS NL72 rack, the AMD Helios rack, and the Google Ironwood rack are all examples of the same rack-as-unit-of-scale architecture, with each vendor designing the rack as a fully engineered unit and with each rack being the principal unit at which a single AI workload is deployed. The architectural convergence on the rack-as-unit-of-scale architecture is one of the most consequential single developments in the data-centre industry, and the convergence is being driven by the AI compute workload’s needs rather than by any single vendor’s design choices.

The data-centre campus layer

The data-centre campus layer of the chip-to-grid system is the layer at which the rack-scale systems, the superpod-scale systems, the cooling infrastructure, the power infrastructure, the network infrastructure, and the security infrastructure are integrated into a single engineered campus. The campus layer has been, in late 2026, the second-most-consequential architectural change in the data-centre industry, and the campus layer’s design is being driven by the AI compute workload’s needs for power, for cooling, for network, and for security at a scale that the conventional data-centre architecture was not designed for.

The most important campus-level change is in the power infrastructure. The conventional data-centre campus of the early 2020s drew power from the grid through a single substation and through a relatively simple distribution network. The AI data-centre campus of the late 2020s is being designed with substantially more sophisticated power infrastructure, with the campus drawing power from multiple substations, with on-site power generation in many cases, with substantial battery-storage capacity in many cases, and with sophisticated power-management software that shifts the data-centre load across the available power sources in response to the price and the availability of the power. Microsoft’s planned multi-gigawatt Maia deployment, the Anthropic 5-gigawatt TPU and AWS deployment, and the OpenAI gigawatt-scale deployment of Jalapeño silicon are all campus-level projects that require power infrastructure at a scale that the conventional data-centre architecture was not designed for.

The second most important campus-level change is in the cooling infrastructure. The conventional data-centre campus of the early 2020s used air cooling for the bulk of the data-centre load, with water cooling used only for the highest-density racks. The AI data-centre campus of the late 2020s is being designed with substantially more sophisticated cooling infrastructure, with the campus using liquid cooling for the bulk of the data-centre load, with direct-to-chip liquid cooling for the highest-density racks, and with immersion cooling for the highest-density racks in many cases. The cooling architecture is, in turn, being designed around the AI compute workload’s needs for cooling at the rack level and at the superpod level.

The third most important campus-level change is in the network infrastructure. The conventional data-centre campus of the early 2020s had relatively modest connectivity, with the campus connected to the broader internet through a relatively small number of long-haul fibre links. The AI data-centre campus of the late 2020s is being designed with substantially more sophisticated network infrastructure, with the campus connected to the broader internet through multiple long-haul fibre links, with substantial long-haul fibre capacity for the campus-to-campus connectivity that the AI compute workload requires, and with substantial local connectivity for the edge-of-network deployments that the AI compute workload is increasingly using.

The grid connection as the unit of energy

The grid connection is, in the chip-to-grid design philosophy, the layer at which the AI data-centre campus connects to the broader electric grid. The grid connection has been, in late 2026, the most rapidly evolving layer of the chip-to-grid system, and the grid-connection design is being driven by the AI compute workload’s needs for substantial power, for low-cost power, and for high-quality power. The grid-connection design is also being driven by the power-wall constraint that the AI compute buildout has produced, with the grid connection becoming a constraint on the AI compute buildout’s pace rather than a passive source of the AI compute buildout’s power.

The most visible architectural change in the grid-connection layer is the move toward multi-substation power delivery. The conventional AI data-centre campus of the early 2020s drew power from a single substation, with the grid connection being a single point of failure and a single point of capacity constraint. The AI data-centre campus of the late 2020s is being designed with multi-substation power delivery, with the campus drawing power from two or more substations and with the data-centre load able to be shifted across the substations in response to the price and the availability of the power. The multi-substation design is, in turn, allowing the AI data-centre campus to participate in the grid’s capacity markets and in the grid’s demand-response markets in ways that the single-substation design could not.

The second architectural change in the grid-connection layer is the move toward on-site power generation. The conventional AI data-centre campus of the early 2020s drew all of its power from the grid. The AI data-centre campus of the late 2020s is being designed with substantial on-site power generation in many cases, with the on-site generation including natural-gas turbines, fuel cells, solar panels with battery storage, and small modular reactors in some of the largest campuses. The on-site power generation allows the AI data-centre campus to be partially insulated from the grid power-outage risk and from the grid power-price volatility, and the on-site generation can, in some cases, allow the AI data-centre campus to export power to the grid during periods of low data-centre load.

The third architectural change in the grid-connection layer is the move toward sophisticated power-management software. The conventional AI data-centre campus of the early 2020s had a relatively simple power-management system, with the campus drawing a roughly constant load from the grid. The AI data-centre campus of the late 2020s is being designed with substantially more sophisticated power-management software, with the software shifting the data-centre load across time, across locations, and across power sources in response to the price and the availability of the power. The power-management software is, in some cases, sophisticated enough to allow the AI data-centre campus to participate in the grid’s frequency-regulation markets, with the campus load being modulated at a sub-second timescale in response to the grid’s frequency deviations.

The chip-to-grid system and the longer-term trajectory

The chip-to-grid system that the AI data-centre industry is designing in 2026 is, in substantive terms, the most consequential change in the data-centre industry’s design principles since the emergence of the internet data centre in the late 1990s and the early 2000s. The chip-to-grid design philosophy is being driven by the AI compute workload’s needs for substantial power, for substantial cooling, for substantial network, and for substantial computational density at every layer of the stack. The chip-to-grid design philosophy is also being driven by the power-wall constraint that the AI compute buildout has produced, with the chip-to-grid design being the response to the constraint.

The most consequential longer-term implication of the chip-to-grid design philosophy is that the AI data-centre industry is, by late 2026, no longer a building-construction industry. The AI data-centre industry is, instead, becoming a fully engineered-systems industry, with the data-centre campus being engineered from the silicon to the grid connection rather than being assembled from conventional building components and conventional server components. The fully engineered-systems approach allows the data-centre campus to be more efficient, more reliable, and more capable than the conventional assembly approach could produce. The fully engineered-systems approach is also more capital-intensive and more complex than the conventional assembly approach, with the implication that the data-centre industry is being concentrated into the hands of the largest hyperscalers and the largest neoclouds.

The second most consequential longer-term implication of the chip-to-grid design philosophy is that the AI compute silicon, the data-centre rack, the data-centre campus, and the grid connection are being designed together rather than separately. The co-design is, in turn, allowing the AI compute buildout to proceed at a pace that the conventional separately-designed approach could not. The co-design is also allowing the AI compute buildout to take advantage of architectural opportunities (such as the rack-level interconnect, the superpod-level network, and the campus-level cooling) that the conventional approach could not exploit.

The third most consequential longer-term implication of the chip-to-grid design philosophy is that the chip-to-grid system is becoming the principal organizing principle for the AI compute industry’s evolution over the rest of the decade. The chip-to-grid system is, in substantive terms, the integration of all of the major architectural themes that the AI compute industry is going through: the rack-as-unit-of-scale architecture, the superpod-as-unit-of-aggregation architecture, the campus-as-unit-of-infrastructure architecture, and the grid-as-unit-of-energy architecture. The integration is being driven by the AI compute workload’s needs, and the integration is, in turn, reshaping the design principles for the AI compute industry.

The chip-to-grid system is, in summary, the most consequential single architectural change in the data-centre industry since the emergence of the internet data centre. The chip-to-grid system is being designed at every layer of the stack — the silicon layer, the rack layer, the data-centre layer, and the grid layer — and the integration of the layers is reshaping the design principles for the AI compute industry. The chip-to-grid system is going to be the principal organizing principle for the AI compute industry’s evolution over the rest of the decade, and the chip-to-grid system is going to be the principal determinant of the AI compute industry’s economics, performance, and strategic position.