The Infrastructure Challenges of AI Data Centers and How AI Could Help Manage Them
The same workloads pushing racks toward a megawatt and grids toward their breaking point are also driving a wave of emerging tools to cool facilities, steady utilities, and shift compute toward cleaner power. Most are early, but the direction is clear.
By Sharmila Bokka, Sr. Electrical Engineer, Crusoe
The 2026 OCP Global Summit organized much of its facility and energy programming into two linked tracks, AI Open Data Center and AI Open DC: Power Distribution. One theme ran through both: the AI workload is reshaping the data center from the chip outward, and the same intelligence is being turned back on the infrastructure to manage it. This article follows that arc, from rack to grid to the global energy system, and points to the OCP workstreams, specifications, and Summit sessions already closing the gaps.
The shift from CPU to GPU systems changed the data center at the level of physics. A general-purpose rack drew a few kilowatts and was air-cooled; a rack of AI GPUs draws tens to hundreds of kilowatts and cannot be air-cooled at all. The effect compounds outward: denser racks force liquid cooling and higher-voltage delivery, larger facilities strain the local utility, and enough facilities in one region begin to affect grid stability. What begins as a chip-level decision ends as an energy-supply problem for an entire market.
What makes this manageable is that the same machine learning driving the load can also govern the infrastructure that carries it. Predictive algorithms turn static infrastructure into responsive environments: cooling that anticipates heat, load management that shapes when and where deferrable work runs, and scheduling that aligns compute with the cleanest available power. Used this way, the data center shifts from an unpredictable consumer toward a grid-cooperative asset.
The problem
A single high-end GPU now draws 1,000 to 1,200 W, and that figure rises each generation as advanced packaging concentrates more heat into smaller hotspots [1]. At the rack the totals are stark: an H100-class rack draws roughly 40 kW, a Blackwell GB200 rack about 120 kW, and Rubin-generation “Kyber” racks are projected at 600 kW to 1 MW by 2027 [2].
1. Power density and heat at the rack
That trajectory forces the move to higher-voltage delivery, a central thread of Summit’s AI Open DC: Power Distribution programming. In the keynote “Shifting to DC Power Distribution in Data Centers,” ABB’s Paolo Catapane and Eaton’s Peter De Bock traced the escalation from 208 V single-phase to 480 V three-phase to 800 VDC, arguing that direct-current distribution is the proven way to move this much power into a small space without an unmanageable copper burden.
Air cannot carry away heat flux at this level, so direct-to-chip liquid or immersion cooling becomes mandatory. But that only makes the heat removable, not cheap to remove: a plant tuned to a fixed worst case overcools the hall whenever real load is lower, which is most of the time. The problem is as much economic as thermal: holding every chip within limit while spending the least energy as load shifts minute to minute.
The response: predictive and zonal cooling
AI governs how efficiently that baseline runs. In place of fixed setpoints, dense sensor networks feed models that forecast hotspots hours ahead and adjust fan speeds, valves, and chilled-water flow before conditions degrade [3], and direct cooling to the specific aisles that need it rather than holding a whole hall cold. The evidence is substantial: reinforcement-learning agents co-optimizing chillers, flow, and fans cut cooling energy by roughly 40 percent in one hyperscale deployment, moving PUE toward 1.0 [12].
Meta’s “Managing Liquid Cooling at Scale” made the operational version of the point: once liquid cooling is in the fleet, the hard part is the tooling and telemetry around it, instrumenting valves and manifolds and automating the response under one framework, which is the same substrate those predictive models depend on.
Where OCP work connects. This higher-voltage delivery is the subject of OCP’s DCF Power Distribution LVDC white paper (v1.1.0), developed in the Power Distribution subproject of Data Center Facility, where rack-level power delivery toward 800 VDC is being standardized.
2. Load volatility and stability at the grid
The problem
A gigawatt-scale campus is not only large, it is volatile, and the volatility is the harder problem. As tens of thousands of GPUs synchronize, checkpoint, and fall idle together, the load can rise or collapse by a large fraction in seconds, showing up on the grid as frequency and voltage excursions that, without fast compensation, can cascade [4]. It is the speed and depth of the swings, not the magnitude, that threatens stability.
Summit’s AI Open Data Center track framed this in utility terms. In “Grid Interconnection Requirements for Large Loads,” NVIDIA’s Sai Somayajula and Mike Tu, with Equinix’s Eric Mannarino, described AI campuses as capacitive, switched power-electronic loads rather than the steady resistive loads the grid was planned around, and walked through emerging ride-through mandates (about 500 milliseconds in Ireland, one to two seconds in Finland) to be demonstrated by simulation before energization. The gap worth closing is standardizing how that behavior is characterized in the first place.
The response: an AI control layer that smooths the draw
The swings track training schedules and job mixes, so they can be forecast, and a forecastable load can be smoothed. Models shift the deferrable tier of batch and non-urgent compute into the troughs, so the net draw rises and falls more gently: peak reductions of 15 to 25 percent are reported in large facilities [7], and during grid-stress events a facility can ease deferrable load back entirely [8]. NVIDIA’s Sean James made the operator’s case in “Flexibility the Grid Can Count On”: making data centers flexible lets them tap interconnection capacity already on the grid rather than waiting on new build-out.
Scheduling cannot catch the fastest transients, so the control layer pairs it with storage. Battery energy storage configured as the UPS absorbs the millisecond steps and presents a smoother draw upstream, supplying the ride-through high-density systems require [13]. That grid-aware role was the subject of “Path to the Megawatt Rack,” where Echion Technologies’ Tyler Andrews and Proper Voltage’s Drew Schaller presented a symmetric, high-power UPS built on high charge-rate chemistry for exactly this duty.
Where OCP work connects. The characterization gap is being closed directly: Google’s Mike Della Penna and Ashish Upreti presented the version-one Interface Specification Between Hyperscalers and Utility Grid for Disturbance Performance (recording), co-signed by Meta and NVIDIA, covering voltage ride-through, active-power recovery, and facility coordination. On the storage side, OCP-Ready Requirements for Energy Storage Systems and the ESS workstream in the new Energy project define the systems that buffer those transients, the subject of EPC Power’s Jason Barmann in “Enabling Grid Compliant Data Center Growth Through Advanced Energy Storage.”
3. Carbon intensity at the global scale
The problem
At the system level the load is large and growing fast: the IEA projects data-center electricity use roughly doubling to about 945 TWh by 2030, near 3 percent of global demand [9], and NERC’s 2025 Long-Term Reliability Assessment raised its ten-year US summer peak-demand forecast by about 224 GW, attributing most of it to new data centers [5]. Total demand growth is a generation and policy problem, but a narrower one is in the operator’s control: when and where that demand lands. At Summit, Google’s Amber Huffman and Tom Garvens made it concrete in “From Chip to Grid,” describing how utilities hold ample capacity but release it on strict terms (the firm-versus-non-firm distinction, where an operator takes interruptible supply the utility can call back), so where a gigawatt load lands is negotiated against the grid, not assumed.
Data-center load draws what it needs whether the power on the wire is wind or coal. The growth is geographically concentrated and often outpaces new renewable supply, so utilities extend fossil plants and add gas to serve it [6], and a clean-capable demand is met partly by fossil fuels because it runs at the wrong hours and sits in the wrong places [10]. The answerable problem is the load’s carbon intensity: its poor alignment in time and place with the clean power already available.
The response: carbon-aware computing
The fix is scheduling (when each job runs and where), which is exactly the control AI is suited to. A model trained on carbon-intensity, load, and price forecasts can push work to the cleanest hours and site new runs where clean power is abundant. Google’s carbon-intelligent platform is the clearest production example, shifting flexible jobs in time, then between regions to follow hourly carbon-free energy [11]. In the same “From Chip to Grid” session, Google framed the endpoint: an operator not only drawing from the grid but participating in it, aiming to match its consumption to carbon-free energy.
For frontier training this is still early: flexibility revenue is small next to compute revenue, so operators default to running flat out. The clearest evidence training can flex is a March 2026 demonstration at Nebius’s London AI Factory (Emerald AI with National Grid, EPRI, and NVIDIA), showing AI infrastructure as a grid-responsive asset without disrupting critical work [14]. One caveat for engineers: shifting on price alone can raise emissions, since off-peak hours on a fossil-heavy grid are often dirtier. Scheduling has to key on carbon intensity directly [15].
Where OCP work connects. Coordinating power, cooling, and the OT management and telemetry interfaces that make a facility grid-responsive is the remit of the Open Data Center for AI strategic initiative, launched in response to the open-letter call for collaboration from Google, Meta, and Microsoft, the framing behind the grid-oriented talks in the AI Open Data Center track.
Conclusion
At each level, from rack to grid to the global energy system, the same workloads that create the problem supply the means to manage it: predictive cooling holds high-density racks within limit; load shaping and demand response turn a volatile campus into a grid-friendly load; and carbon-aware scheduling aligns compute with the cleanest generation. Solving all three at once is a single chip-to-grid co-design challenge, the framing that ran through the Summit sessions above. It all complements the underlying work (liquid cooling at scale, 800 VDC, interconnection reform, and firm clean supply remain prerequisites), and the flexibility that protects a data center’s own uptime is exactly what grids need as they add wind and solar. Managed deliberately, AI data centers could become one of the largest sources of flexible demand on the grid, advancing a cleaner, more reliable energy system rather than working against it.
Join the conversation
The facility and energy side of OCP is where an electrical engineer can contribute directly. The Power Distribution subproject and Energy project within Data Center Facility, and the broader Open Data Center for AI initiative, all run open calls and mailing lists; the OCP Community pages show how to get involved, and every talk above is in the 2026 Global Summit recorded content.
To go deeper, the free OCP Academy turns many of these contributions into short courses. The ones that track this article include LVDC Power Distribution for Data Centers, Solid State Transformers: Medium Voltage to 800 VDC, Grid Disturbance Performance for Hyperscale Data Centers, OCP-Ready Requirements for Energy Storage Systems, and Open Data Center for AI: Applying Specification 0.7.
References
- Sze, V., et al. “Hardware for Machine Learning: Challenges and Opportunities.” MIT.
- “NVIDIA 800 VDC Architecture Will Power the Next Generation of AI Factories,” NVIDIA Technical Blog; and “Preparing for 800 VDC Data Centers,” Data Center Frontier (rack-power progression H100 → GB200 → Rubin Kyber).
- Ma, S., et al. (2025). “Artificial-intelligence-enabled predictive energy-saving planning based on a Transformer-GRU model for coolant-temperature prediction.” Applied Energy, 389, 124576.
- Electric Power Research Institute (EPRI), large-load grid-integration research. epri.com
- North American Electric Reliability Corporation, 2025 Long-Term Reliability Assessment (2025); see also Utility Dive and Power Engineering coverage of the roughly 224 GW summer-peak / 69 percent revision and 13-of-23 at-risk regions.
- U.S. Energy Information Administration, Annual Energy Outlook. eia.gov/outlooks/aeo
- “AI-Enhanced Demand Response Strategies in Smart Grids: Toward a Sustainable Energy Future” (2024).
- Bashyal, A., et al. (2024). “Artificial-intelligence-enabled dynamic demand-response system for maximizing the use of renewable electricity in production processes.” The International Journal of Advanced Manufacturing Technology, 138(1), 247–271.
- International Energy Agency, Energy and AI (2025) and Key Questions on Energy and AI (2026).
- U.S. Department of Energy, report evaluating the increase in electricity demand from data centers. energy.gov
- Radovanovic, A., Koningstein, R., et al. (2021). “Carbon-Aware Computing for Datacenters.” IEEE Transactions on Power Systems; and Hall, S., Micheli, F., & Radovanovic, A. (2025), “Carbon-Aware Computing for Data Centers with Probabilistic Performance Guarantees,” arXiv:2410.21510.
- Evans, R. & Gao, J. (2016). “DeepMind AI Reduces Google Data Centre Cooling Bill by 40%.” DeepMind.
- “Data Center World 2026: AI Pushes Infrastructure to New Limits,” Data Center Knowledge (energy storage for power quality and voltage ride-through); see also Goldman Sachs, “Fuel Cells Could Help Meet the Power Demand from Data Centers” (2025), on behind-the-meter generation.
- Emerald AI with National Grid, EPRI, and NVIDIA, London AI Factory (Nebius) demonstration of grid-responsive AI compute, March 2026; see also Bessemer Venture Partners, “Roadmap: The AI Data Center Stack” (2026).
- Knittel, Senga & Wang (2025), on price- versus carbon-based load shifting and the risk of higher emissions on fossil-heavy grids; via Green Software Foundation, “Data Centers as Flexible Grid Participants” (2026).
Further reading
• Hennessy, J. & Patterson, D. Computer Architecture: A Quantitative Approach, 6th ed.
• “Transmission Lines Will Determine America’s Power Future,” POWER Magazine.
• “Virginia Faces New Headwinds in Data Center Growth,” Data Center Knowledge.
• Deloitte, 2025 Power and Utilities Industry Outlook.
OCP Community references
• 2026 OCP Global Summit tracks: AI Open Data Center and AI Open DC: Power Distribution.
Summit talks referenced:
• Shifting to DC Power Distribution in Data Centers (keynote), Paolo Catapane (ABB), Peter De Bock (Eaton).
• Managing Liquid Cooling at Scale, Kalpak Dhake and Zach Miller (Meta).
• Grid Interconnection Requirements for Large Loads, Sai Somayajula and Mike Tu (NVIDIA); Eric Mannarino (Equinix); Joshua Piruzza (Danovo Energy Solutions).
• Flexibility the Grid Can Count On, Sean James (NVIDIA).
• Grid Requirements and Mitigation Options for Large-Scale AI/ML Loads (disturbance-performance spec), Mike Della Penna and Ashish Upreti (Google).
• Enabling Grid Compliant Data Center Growth Through Advanced Energy Storage, Jason Barmann (EPC Power).
• Path to the Megawatt Rack, Tyler Andrews (Echion Technologies), Drew Schaller (Proper Voltage).
• From Chip to Grid, Amber Huffman and Tom Garvens (Google).
OCP specifications and white papers:
• Base Spec: Interface Specification Between Hyperscalers and Utility Grid for Disturbance Performance (2026).
• OCP-Ready Requirements for Energy Storage Systems.
• DCF Power Distribution LVDC, version 1.1.0.
OCP Academy courses (free):
• LVDC Power Distribution for Data Centers; Solid State Transformers: Medium Voltage to 800 VDC; Grid Disturbance Performance for Hyperscale Data Centers; OCP-Ready Requirements for Energy Storage Systems; Open Data Center for AI: Applying Specification 0.7. Browse the full catalog in the OCP Data Center Technologies track.