For decades, data center cooling was primarily a facilities consideration.
With modern AI infrastructure, it has become part of the compute architecture itself.
An NVIDIA H200 SXM GPU can operate at up to 700W TDP, while newer Blackwell systems push power density significantly higher. At the extreme end, an NVIDIA GB200 NVL72 rack combines 72 Blackwell GPUs and 36 Grace CPUs in a liquid-cooled rack-scale architecture. NVIDIA documentation puts a fully loaded GB200 NVL72 rack at approximately 120 kW.
At those densities, choosing between air and liquid cooling is no longer simply a question of cooling efficiency.
It affects:
- rack density
- power distribution
- data center design
- deployment cost
- server selection
- maintenance
- scalability
- and ultimately how much AI compute can be deployed in a given facility
So where does traditional air cooling still make sense, and when does liquid cooling become necessary?
Why AI Servers Generate So Much Heat
Almost all electrical power consumed by a server eventually becomes heat that must be removed from the data center.
A conventional enterprise server may consume hundreds of watts or a few kilowatts.
An 8-GPU AI server is different.
Consider an 8-GPU H200 SXM configuration.
With each H200 SXM configurable up to 700W, the GPUs alone can represent as much as:
8 × 700W = 5.6 kW
And that is before adding:
- CPUs
- DDR5 memory
- NVMe drives
- NICs and DPUs
- NVSwitch components
- fans
- power-conversion losses
This is why the thermal characteristics of AI servers differ dramatically from conventional CPU servers. NVIDIA specifies H200 SXM at up to 700W TDP, with 141GB of HBM3e and 4.8TB/s of memory bandwidth per GPU.
How Air-Cooled AI Servers Work
Air cooling remains the most familiar data center architecture.
Server fans pull cool air through the front of the chassis. The air passes across heatsinks attached to GPUs, CPUs, memory and other components before hot air exits through the rear.
A typical airflow path is:
Cold aisle → Server fans → CPU/GPU heatsinks → Hot aisle
The major advantage is simplicity.
Existing data centers are already designed around this model, which means an organization may be able to deploy an air-cooled AI server without installing an entirely new cooling infrastructure.
Advantages of Air Cooling
Air-cooled systems offer several practical benefits:
- mature infrastructure
- easier installation
- familiar maintenance procedures
- no coolant inside the server
- simpler rack integration
- broad compatibility with existing data centers
For small and medium AI deployments, those advantages can outweigh the density benefits of liquid cooling.
H200: Air Cooling Is Still Very Relevant
H200 represents an interesting point in the cooling transition.
NVIDIA offers the H200 in both SXM and NVL configurations.
The H200 SXM has a maximum configurable TDP of up to 700W, while H200 NVL is specified at up to 600W and uses a dual-slot, air-cooled PCIe form factor.
This means organizations can still deploy significant H200 compute capacity using conventional air-cooled infrastructure.
For many enterprise environments, this is important.
If a company already operates a data center designed for high-density servers, deploying air-cooled H200 systems may require considerably less infrastructure modification than moving directly to liquid cooling.
However, as more high-power GPU servers are placed into the same rack, the problem moves from cooling an individual server to cooling the entire rack.
The Real Problem Is Rack Density
Cooling should not be evaluated only at the GPU level.
Imagine several high-power 8-GPU systems installed in the same rack.
Even if each individual server can technically operate with air cooling, the combined thermal load may exceed what the rack or facility was designed to handle.
This creates several constraints:
Airflow
Thousands of cubic feet of air may need to move through a densely populated rack.
Fan Power
Higher airflow requires faster fans, increasing server power consumption.
Facility Cooling
CRAC or CRAH systems must remove the additional heat from the room.
Rack Density
Operators may intentionally leave rack positions empty simply because the facility cannot cool a fully populated rack.
At that point, the limiting resource isn't rack space.
It's heat-removal capacity.
B200 Changes the Equation
Blackwell pushes AI infrastructure further toward high-density computing.
NVIDIA's HGX B200 platform integrates eight Blackwell SXM GPUs with fifth-generation NVLink and NVLink Switch technology. The platform provides 1.4TB of total GPU memory and 1.8TB/s of GPU-to-GPU NVLink bandwidth per GPU.
But importantly, deploying B200 does not automatically mean liquid cooling.
There are B200 server architectures designed around high-performance air cooling as well as systems designed for liquid-cooled environments.
This gives infrastructure teams a choice.
For organizations with existing data centers, air-cooled B200 servers can provide a migration path into Blackwell without immediately rebuilding facility cooling.
For new AI factories and very dense deployments, however, liquid cooling becomes increasingly attractive.
Why Liquid Cooling Is More Efficient
Air is not an especially efficient medium for moving large quantities of heat.
Liquids have much greater heat capacity and can transport thermal energy far more effectively.
In a direct liquid cooling system, coolant flows through cold plates attached directly to high-power components such as GPUs and CPUs.
The thermal path becomes:
GPU/CPU → Cold plate → Coolant → CDU → Facility water loop
Instead of moving enormous volumes of air through the server, heat is transferred directly into a liquid loop.
What Is a CDU?
A critical component in many liquid-cooled deployments is the Coolant Distribution Unit (CDU).
The CDU sits between the server cooling loop and the facility cooling infrastructure.
Its responsibilities can include:
- circulating coolant
- controlling flow rates
- managing pressure
- monitoring temperature
- transferring heat to the facility water system
- detecting cooling faults
Depending on the deployment, CDUs may be installed at the rack, row, or facility level.
This means purchasing liquid-cooled AI servers isn't just a server procurement decision.
The facility must support the cooling architecture surrounding them.
GB200 NVL72: Cooling Becomes Part of the Architecture
The transition becomes especially clear with NVIDIA GB200 NVL72.
Unlike a conventional standalone GPU server, GB200 NVL72 is designed as a rack-scale system.
A single system integrates:
- 72 Blackwell GPUs
- 36 Grace CPUs
- 13.4TB of HBM3e GPU memory
- 17TB of CPU memory
- 130TB/s aggregate NVLink bandwidth
All 72 GPUs participate in one large NVLink domain.
This architecture produces extraordinary compute density—and an equally extraordinary thermal challenge.
NVIDIA's Open Compute Project design work specifies approximately 120 kW of cooling capacity for the rack and uses direct liquid cooling with liquid cooling manifolds and blind-mate connections.
At this point, liquid cooling isn't simply an optimization.
It is part of the system architecture.
From Server-Level to Rack-Level Computing
GB200 also illustrates a broader change taking place in AI infrastructure.
Historically, procurement teams purchased servers.
Increasingly, they are purchasing compute racks.
Instead of asking:
How many GPUs fit in this server?
Infrastructure teams increasingly need to ask:
How much compute can this facility support per rack?
That changes the procurement conversation dramatically.
The server cannot be evaluated independently from:
- rack power
- PDUs
- busways
- cooling loops
- CDUs
- networking
- floor loading
- facility water temperature
- redundancy requirements
AI infrastructure is becoming a system-level engineering problem.
Air Cooling vs Liquid Cooling
Neither technology is universally better.
The correct choice depends on deployment scale and facility capability.
When Air Cooling Still Makes Sense
Air cooling remains attractive when:
- deploying a small number of GPU servers
- working within an existing enterprise data center
- rack density is moderate
- facility modifications would be expensive
- fast deployment is important
- the selected server is specifically validated for air cooling
H200 and air-cooled Blackwell servers can therefore remain very practical options.
Liquid cooling should not be adopted simply because it is newer.
If an existing facility can reliably provide the required airflow, power and ambient conditions, air cooling may deliver a lower-complexity deployment.
When Liquid Cooling Makes More Sense
Liquid cooling becomes increasingly compelling when:
- deploying many high-power GPUs
- maximizing GPUs per rack
- building new AI factories
- operating Blackwell rack-scale architectures
- facility floor space is constrained
- reducing fan power is important
- future GPU generations are part of the infrastructure roadmap
For these environments, investing in liquid infrastructure can provide a foundation for multiple generations of increasingly dense accelerators.
Cooling Is Only Half the Problem
A 100+ kW AI rack also creates a major electrical engineering challenge.
The facility needs to deliver that power reliably.
This can affect:
- rack PDUs
- breakers
- busways
- UPS systems
- transformers
- backup generation
- power redundancy
For GB200 NVL72 specifically, NVIDIA's administration documentation uses approximately 120 kW at full rack load, including compute nodes and rack components.
That is dramatically different from deploying traditional enterprise racks.
Power and cooling therefore need to be designed together.
What Procurement Teams Should Ask Before Buying
Before ordering a high-density AI system, procurement teams should verify more than GPU model and quantity.
Important questions include:
What is the maximum server power draw?
Not just GPU TDP.
Is the configuration air cooled or liquid cooled?
Different configurations of the same GPU generation may have very different infrastructure requirements.
What inlet temperature and airflow are required?
An air-cooled server is useful only if the facility can actually supply the necessary airflow.
Does liquid cooling require a CDU?
And if so, is it included with the system?
What coolant and facility-water specifications are required?
These requirements must match the data center.
What power infrastructure is required at rack level?
The answer may determine whether the system can be installed at all.
What networking is required?
Dense AI systems can also require multiple 400G or 800G network interfaces, creating additional power and thermal load.
Cooling Should Be Evaluated Before the Purchase Order
One of the most expensive mistakes in AI infrastructure procurement is selecting the compute hardware first and asking facilities teams about power and cooling afterward.
The process should happen in reverse.
Before choosing the system, organizations should determine:
- Available power per rack
- Available cooling capacity
- Maximum practical rack density
- Airflow or liquid-loop capability
- Network infrastructure
- Physical rack requirements
- Expansion plans
Only then should the final server architecture be selected.
The Bigger Picture
H200, B200 and GB200 illustrate three stages in the evolution of AI infrastructure.
H200 can still fit naturally into many conventional high-performance data centers.
B200 pushes compute and thermal density significantly higher while offering deployments that can bridge existing and next-generation infrastructure.
GB200 NVL72 moves the industry toward integrated, liquid-cooled rack-scale computing where power, cooling, networking and compute must be designed as one system.
The question is therefore no longer simply:
Air cooling or liquid cooling?
The more important question is:
How much AI compute does your facility need to support—and at what density?
Final Thoughts
Cooling has become one of the defining architectural decisions in modern AI infrastructure.
Air-cooled systems remain practical for many H200 and Blackwell deployments, particularly when organizations want to use existing data center infrastructure.
But as GPU power and rack density increase, liquid cooling allows organizations to deploy considerably more compute within the same physical footprint.
At GB200 NVL72 scale, the transition is even more fundamental: cooling becomes part of the rack-scale compute architecture itself.
For procurement teams, that means evaluating the GPU, server, power infrastructure and cooling system together—not as separate purchasing decisions.
Planning an AI infrastructure deployment?
REVO.tech helps organizations source new, excess-inventory, and professionally tested enterprise AI hardware, including NVIDIA H100, H200 and Blackwell platforms, complete AI servers, networking, storage and supporting data center hardware.
Contact [email protected] to discuss current requirements, configurations and availability.