How AI is reshaping data center design

Traditional designs built for 5-8 kW per rack cannot support AI workloads that require 40 to 600 kW per rack, forcing fundamental redesigns of power infrastructure, cooling systems, and location strategy.

How do AI workloads differ from traditional IT?

AI workloads create demands that legacy infrastructure cannot meet. GPUs require 10 to 15 times as much energy as traditional CPUs because they contain more transistors in their arithmetic logic units. AI workloads create demands that legacy infrastructure cannot meet. GPUs require 10 to 15 times more energy than traditional CPUs because they contain more transistors in arithmetic logic units. NVIDIA's Blackwell GPUs show a 4.8x power increase over the previous generation.

AI has two distinct phases with different needs:

Training:

• Requires massive, concentrated compute power
• High-bandwidth interconnects between GPUs
• Less latency-sensitive
• Can locate where power and space are abundant

Inference:

• Requires geographic distribution near users
• Low latency for real-time applications
• Must integrate with existing cloud infrastructure

Structure Research projects inference will overtake training in 2026, ultimately representing 80% of total AI capacity by 2030. This is a reversal from 2023's 95% training dominance.

Read our blog: Will AI eat the cloud?

Forrester Research identifies AI as creating an entirely new computing ecosystem with a five-layer stack spanning experience, orchestration, data, intelligence, and infrastructure layers. This means data centers must support massive data pipelines, multiple model types, and integration with at least eight major provider categories—from chipmakers to model builders to hardware OEMs.

How is power density and electrical infrastructure changing?

Power availability has emerged as the single most critical constraint in AI data center design. Gartner projects data center electricity consumption will rise from 448 TWh in 2025 to 980 TWh by 2030, with AI-optimized servers growing from 93 TWh to 432 TWh—nearly fivefold. By 2030, AI servers will represent 64% of incremental power demand. In the United States alone, data center consumption will increase from 4% to 7.8% of regional electricity use.

Traditional data centers designed for 5-8 kW per rack cannot support modern AI. Current workloads require 30-50 kW per rack baseline, with leading-edge deployments pushing toward 120 kW. Future systems like NVIDIA's Vera Rubin will require up to 600 kW per rack. For the top four cloud providers, power demand jumped from 910 MW in 2024 to 4,320 MW in 2025—a 4.8x increase in one year.

This requires rethinking every power element: utility connections, substation capacity, backup power with low-carbon alternatives, power distribution units sized for 10x previous loads, and sophisticated monitoring. Gartner's November 2024 warning identified three critical impacts: 40% of existing AI data centers will be limited by power availability by 2027; costs will increase significantly as operators compete for capacity; and surging demand is forcing utilities to keep fossil fuel plants operational beyond scheduled shutdowns, increasing short-term CO2 emissions.

What cooling challenges are introduced by AI hardware?

Power density drives heat density, making cooling a defining design challenge. McKinsey estimates cooling accounts for nearly 40% of total data center energy consumption—a major cost and sustainability factor.

The industry is rapidly progressing through cooling architecture:

• Legacy air cooling (< 8kW/rack): Traditional workloads only
• Modern air cooling (< 30kW/rack): Practical limits reached
• Active Rear Door Heat Exchangers (30-70kW/rack): H100 GPUs at 40kW/rack
• Direct-to-Chip Liquid Cooling (50-200kW/rack): GB200 GPUs at 120kW/rack
• Liquid Immersion Cooling (250kW-1MW/rack): Future Vera Rubin at 600kW/rack

Liquid cooling offers dramatically greater efficiency but introduces operational complexity. Even minor leaks can cause catastrophic hardware failure, requiring advanced leak detection, automated shutoff valves, and specialized maintenance expertise. The thermal challenge extends beyond capacity to response speed—traditional data centers had two to three hours to address thermal runaway; high-density AI environments have only minutes.

Real-world results demonstrate impact: BMO Bank deployed Siemens' AI-driven cooling optimization and reduced energy consumption by 55%. Foxconn used digital twin simulation and reduced consumption by 30%. These combine advanced cooling hardware with AI-driven predictive systems that simulate failures before they impact operations.

Why does location matter more for AI data centers?

The shift from training to inference is fundamentally changing location strategy. Training can locate where resources are abundant—low-cost power, favorable climates, available land—with less concern about latency. Emerging training hubs include Abilene, Texas and Ellendale, North Dakota, offering renewable energy and cold climates.

Inference demands geographic distribution in population centers, low latency for real-time applications, and integration with existing cloud infrastructure. As inference grows to 80% of AI capacity by 2030, proximity to end users becomes critical.

Structure Research projects 2+ GW mega-hubs worldwide by 2030:

North America:

• Northern Virginia: 8.5 GW
• Dallas: 2.8 GW
• Phoenix: 2.7 GW

Europe:

• London: 2.7 GW
• Frankfurt: 2.68 GW
• Paris: 2.0 GW
• Rapid growth in Oslo, Barcelona, Madrid (driven partly by data sovereignty regulations)

Asia Pacific:

• Tokyo: 2.8 GW
• Sydney: 2.4 GW
• Johor: 2.2 GW

This distributed model requires building capacity in multiple markets simultaneously while maintaining consistent capabilities across geographies—more complex and capital-intensive than previous cloud models.

How are operators adapting facility design?

Meeting the 10X challenge requires fundamental shifts in design philosophy across multiple dimensions.

Modular architecture for speed and flexibility

Traditional 18 to 24-month timelines are obsolete. Modular design delivers capacity 60% faster with 13%+ cost reduction. It offers flexibility to adapt as requirements evolve, phased capital deployment, and ability to integrate new technologies without disrupting operations.

At Iron Mountain Data Centers, our Phoenix data centers exemplify this approach, scaling while continuously integrating latest technologies.

AI-driven optimization through digital twins

Digital twin technology enables testing and prediction before problems occur. These virtual models simulate entire platforms with 100,000+ endpoints, predicting cooling and server failures while enabling real-time optimization. This is critical when thermal response timeframes have shrunk from hours to minutes.

Advanced power infrastructure and strategic sourcing

Facilities must secure reliable, 24/7 power at a viable cost with minimal environmental impact. This requires larger electrical systems, strategic utility partnerships, on-site generation capabilities, and battery energy storage systems to balance solar and wind fluctuations. Future integration includes green hydrogen, geothermal, and small modular reactors within 3-5 years.

Sustainable energy leadership

Public scrutiny demands genuine environmental leadership. Iron Mountain Data Centers has offered 100% matched renewable energy since 2017 and is pioneering the 24/7 CFE transition. Over 75% of our consumption is tracked on an hourly basis, with multiple days of 100% carbon-free operation reported at sites with local renewable energy suppliers.

Gartner notes that data centers require reliable 24/7 power, which wind and solar cannot provide alone without storage. Long-term solutions include improved battery storage technologies, clean generation sources like small modular reactors, and waste heat reuse systems.

What does this mean for hyperscale and cloud providers?

For hyperscale and cloud providers, AI infrastructure represents both an enormous opportunity and a significant competitive risk.

Power availability as competitive differentiator

By 2027, 40% of AI data centers will face power limitations, leaving organizations with secured capacity to scale freely, while others face constraints regardless of their capabilities. This requires moving from reactive procurement to strategic utility partnerships, investing in on-site generation and battery storage, exploring emerging alternatives, and securing long-term contracts before costs spike.

Managing architecture and ecosystem complexity

Forrester identifies "agent and architecture sprawl" as a growing challenge. The AI ecosystem now includes eight or more major provider categories with dozens or hundreds of competing vendors. This creates risks of vendor lock-in, nascent quality controls, sovereign AI requirements, and the need to manage hundreds of model versions. Required responses include crafting coherent architectures, selecting smart providers, balancing governance to balance innovation with risk management, and having centers of excellence guide stakeholders through complexity.

Navigating cost pressures and dual architecture requirements

Rising operational costs are inevitable as power shortages drive price increases and competition creates bidding wars. These costs flow through to AI service providers and customers. Providers delivering AI capabilities at lower costs through superior infrastructure efficiency will capture market share.

The training-to-inference shift requires dual architecture: concentrated training capacity in power-rich locations alongside distributed inference capacity in population centers. Maintaining both simultaneously is more complex and capital-intensive than previous cloud models, requiring sophisticated orchestration and careful capacity planning across geographies.

Balancing growth with sustainability and planning for demand

Gartner warns that increased data center use will lead to short-term increases in CO2 emissions as power demand outpaces the availability of clean energy. The tension is real: business growth requires rapid expansion while environmental commitments demand carbon reduction. The path forward requires investing in genuine 24/7 CFE rather than annual renewable credits, deploying battery storage to address renewable intermittency, and partnering with facilities pursuing advanced sustainability strategies.

The cost of Large Language Models with minimum performance benchmarks has decreased approximately 10x annually from 2022 to 2024. According to the Jevons Paradox, as AI becomes more affordable, consumption won't decrease—it will increase. Infrastructure requirements will exceed current projections, making early capacity investment a competitive advantage, while delayed action risks locking out access to increasingly constrained supply.