When we use an AI service, all we see on the screen is the process of entering a question and receiving an answer.
Behind the scenes, however, actual servers and semiconductors are performing computations. In particular, when training large AI models or processing large amounts of computation, multiple GPUs may be used together. NVIDIA also provides dedicated interconnect architectures that connect GPUs at high speed for large-scale AI training and inference.
This is why the term AI Data Center has also become increasingly common.
But this raises a question.
If you install a lot of GPUs in an existing data center, doesn’t it simply become an AI data center?
AI data centers cannot be distinguished solely by the number or proportion of GPUs.
In large-scale AI computing, what matters is that multiple GPUs often divide a single task, perform their parts of the computation, and work together by quickly exchanging their computation results.
That is why, in an AI data center, not only the performance of the GPUs themselves but also the internal network that connects GPU to GPU at high speed is especially important.
As many high-performance devices are operated together, the power and cooling environment they require must also be considered. ASHRAE also treats power and cooling as major design factors in AI data centers.
What Does a Data Center Do?
A data center is a facility where servers, storage devices, network equipment, and other systems are gathered and operated.
The websites and apps we use, corporate business systems, and cloud services can also run on servers in data centers.
Simply placing computers in a facility is not enough to keep servers running.
- Power must be supplied to the equipment reliably.
- Heat generated by the servers must be removed.
- Servers must be connected to one another through networks.
- An environment is needed to operate large amounts of equipment reliably.
Therefore, a data center requires not only servers but also infrastructure such as power, cooling, and networks.
AI data centers also use these basic elements.
The difference is that when multiple high-performance computing devices are used together for AI computation, the way those devices are connected and the environment supporting them may need to change. ASHRAE also explains that in AI data centers, power, cooling, and facility structure should be considered together rather than as separate issues.
Do AI Data Centers Use GPUs Instead of CPUs?
In the previous article, we looked at how GPUs are widely used for AI computation because they are well suited to handling many calculations at the same time.
That does not mean an AI server is a computer where the CPU has been removed and replaced entirely with GPUs.
AI servers can use CPUs and GPUs together.
For example, the CPU can handle various system tasks while the GPU performs large-scale AI computation. Actual data center products designed for AI also combine not only GPUs but CPUs, network equipment, and other components.
Multiple CPUs and GPUs may also be configured within a single server rack.
A server rack is a large metal frame or cabinet used to mount servers and network equipment vertically.
If you have seen photos of data centers, you can think of the large cabinets filled with servers stacked in multiple rows.
What matters is not how many GPUs are installed.
An important starting point for understanding AI data centers is how multiple GPUs are connected when they are used together for one large computation.
Multiple GPUs Work on One Problem Together
General data centers also have many connected servers.
However, each server often performs a different role.
For example, a server that displays a website may request necessary information from a database server.
A simple analogy would be one person serving customers while another person looks up the information they need.
They exchange information, but each person has a different job.
Large-scale AI computation can work somewhat differently.
If an AI model is large or a large amount of computation needs to be processed quickly, multiple GPUs can be brought together to divide one large problem among themselves. NVIDIA’s GPU interconnect technologies are also designed to allow multiple GPUs to communicate quickly with one another during AI training and inference.
Simply put, it is closer to:
“Everyone, do your own job.” becoming “Everyone, come here. We need to solve this one problem together.”
Each GPU takes responsibility for part of a larger computation, and during the process it may need results produced by other GPUs.
So even if one GPU finishes its own calculation, it may have to wait if a result it needs has not yet arrived.
In other words, no matter how fast an individual GPU is, if GPUs are slow at exchanging results with one another, the overall AI computation can also become slower.
That is why, in large-scale AI systems, how quickly multiple GPUs can be connected becomes as important as the performance of an individual GPU.
This Is Why Internal Networks Are Especially Important in AI Data Centers
Networks are also important in general data centers.
User requests must be delivered to servers, and servers with different roles must exchange the data they need.
But when multiple GPUs process one AI computation together, the network takes on another role.
The GPUs must quickly exchange their computation results in order to solve one problem together.
Imagine several people solving one problem together.
Even if each person completes their assigned part quickly, if they are slow to share their results, the others may have difficulty continuing with the next step.
GPUs are similar.
That is why, in an AI data center, not only the network connecting external users to servers but also the internal network that quickly connects GPU to GPU and server to server can directly affect AI computing performance.
NVIDIA, for example, provides NVLink and NVLink Switch to enable high-speed communication among multiple GPUs so that they can operate like one large computing device. In its latest rack-scale systems, dozens of GPUs can also be connected within a single NVLink domain.
There is no need to memorize all of these technology names.
What matters is the difference.
In a general server environment, multiple devices perform their own tasks while exchanging information when needed. In a large-scale AI environment, multiple GPUs can continuously exchange computation results in order to work together on the same task.
So when looking at an AI data center, it is important to consider not only how many GPUs it has, but also how quickly those GPUs are connected to one another.
Running Multiple GPUs Together Also Requires Power
The reason power is frequently discussed in relation to AI data centers is not that GPUs use some special kind of electricity.
Large-scale AI systems can operate many high-performance devices in one place at the same time, including CPUs, GPUs, memory, and network equipment.
As a result, the amount of power required by a single server rack or a certain area can increase.
ASHRAE also explains that AI and high-performance computing environments can concentrate higher power usage in one location than conventional data centers, and recommends considering sufficient power supply capacity from the data center design stage.
Therefore, when adding large amounts of AI equipment to an existing data center, it may not be enough simply to check whether there is physical space for the servers.
You also need to check whether the required power can actually be supplied to that equipment.
However, rather than viewing this as the most important distinction between AI data centers and general data centers, it is better understood as a physical requirement that grows as many high-performance computing devices are operated together.
Running Many Devices Can Also Change the Cooling Method
It cannot simply be said that GPUs are always hotter than CPUs.
What matters is that when many high-performance computing devices consume power in one place, the amount of heat that must be removed from that area can also increase.
In conventional data centers, air cooling, which sends cool air to servers to remove heat, is widely used.
However, if equipment that consumes large amounts of power is concentrated in a small area, situations can arise where air alone cannot remove the heat effectively.
That is why Liquid Cooling is also becoming important in high-density AI systems. ASHRAE also identifies liquid cooling as an important cooling method for high-density AI and high-performance computing environments.
One representative method is Direct-to-Chip Liquid Cooling, which carries coolant close to the GPU or CPU to remove heat.
Not every AI data center must use liquid cooling.
Depending on the equipment being used and how it is arranged, conventional air cooling may still be used.
But as more high-performance equipment is placed in one location, how to remove the heat it generates quickly becomes a more important issue.
Because of this cooling burden, underwater data centers that can use seawater for cooling have also been attracting renewed attention recently. In China, commercial underwater data centers and facilities for AI computing that use seawater for cooling are being built.
In other words, cooling an AI data center is not simply a matter of installing more air conditioners. It is about how to move the heat generated by many computing devices out of the facility.
How Do General Data Centers and AI Data Centers Compare?
The characteristics of an AI data center are not defined by one specific component called the GPU.
In large-scale AI computing in particular, the easiest difference to understand is that a structure that connects multiple GPUs at high speed so they can perform one task together becomes important.
| Category | General Server-Centered Data Center | Data Center for Large-Scale AI Computing |
|---|---|---|
| Role of equipment | Multiple servers handle their own tasks | Multiple GPUs can divide and process one computation |
| Communication between devices | Each server exchanges data when needed | GPUs participating in the same computation may need to exchange computation results quickly |
| Internal network | Used to connect servers and services | GPU-to-GPU connection speed can directly affect AI computing performance |
| Power | Supplied according to the server configuration | High power may be required when many high-performance devices are concentrated in one place |
| Cooling | Air cooling is widely used | Liquid cooling may be required in high-density configurations |
Not all AI data centers use the same structure.
A server running a small AI service and a system connecting many GPUs to train a very large AI model require different environments.
General data centers can also use GPUs, and multiple servers can also divide one task among themselves.
Therefore, there is no absolute boundary between AI data centers and general data centers that can be defined simply by whether GPUs are present.
However, as large-scale AI computation increasingly uses structures in which multiple GPUs work together on the same problem, fast network connections between GPUs become especially important, while the power and cooling requirements needed to support many high-performance devices can also increase.
DANA NOTES in One Line
Just as collective intelligence emerges when several people divide one problem, share their answers, and work together toward a solution, large-scale AI computing divides one problem across multiple GPUs that exchange their results. That is why fast network connections between GPUs are especially important in AI data centers compared with general data centers.
