Even a novice can understand it in seconds: interesting explanation of various core parameters and specifications of GPU!

GPU can be said to be at the center of attention recently.

After many rounds of market rumors, speculation and sentiment reversals, the US government finally upgraded its export controls on Nvidia's H20 chips. Then Nvidia CEO Jensen Huang visited China again after a lapse of three months and expressed his hope to continue cooperating with China. This shows the shock this move caused in the industry.

At present, the most discussed topic is GPU, but I found that everyone is unclear about the core parameters and specifications of GPU cards. Today we will take you step by step to quickly understand the "core parameters and specifications of GPU", such as computing power, video memory (video memory type, video memory bandwidth), inter-card interconnection, power consumption, etc. Let's get started right away!

1. The difference between graphics card and GPU

Graphics cards and GPUs are often mentioned, but their concepts and functions are different.

In simple terms:

The GPU is the core component in the graphics card. It is a dedicated processor designed specifically to perform complex graphics and parallel computing tasks quickly and efficiently. The GPU can be compared to the CPU in a computer, but it is specially optimized for graphics computing.

A graphics card is a complete hardware unit that includes a GPU and is usually installed on the motherboard of a computer. In addition to the GPU, a graphics card also includes multiple components such as video memory, power circuits, and cooling systems. It is an independent hardware device that is responsible for outputting image data processed by the GPU to a display device such as a monitor.

In simple terms, the GPU is the heart of the graphics card, responsible for the core task of graphics processing, while the graphics card is the entire hardware device, including the GPU and other supporting components. The performance of the graphics card depends not only on the capabilities of the GPU, but also on other components such as the video memory and cooling system.

2. Introduction to the meaning of key parameters in GPU

The GPU (Graphics Processing Unit) is responsible for handling all graphics and video output related tasks. Understanding the key parameters of a GPU can help you evaluate its performance and suitability. Here are some important GPU parameters and what they mean:

Computing power: refers to the GPU's ability to perform floating-point operations, usually measured in TFLOPS (floating-point operations per second). High computing power is essential for computationally intensive tasks such as scientific computing, simulation, and deep learning. It can accelerate model training, data analysis, and the processing speed of complex simulations.

Core Count: The number of cores in a GPU is also known as CUDA cores (NVIDIA's term) or stream processors (AMD's term). The number of cores is an important indicator for evaluating the parallel processing capabilities of a GPU. The more cores a GPU has, the better it generally performs at performing graphics and parallel computing tasks.

Core frequency: Also known as the GPU's clock speed, it is measured in MHz (megahertz) or GHz (gigahertz). It indicates how fast the GPU core runs. The higher the frequency, the faster the GPU can process information.

Video memory type : The type of video memory used by the GPU directly affects its performance. Common types include GDDR5, GDDR6, etc., among which GDDR6 provides higher speed and bandwidth than GDDR5.

Video memory capacity: Video memory is the memory used by the GPU to temporarily store image data being processed. The larger the capacity, the better the GPU's ability to handle high-resolution and complex graphics tasks.

Memory bandwidth: This is a measure of the speed at which data can be transferred between the GPU and its memory. The higher the bandwidth, the faster the data exchange and the better the GPU performance.

Manufacturing process: refers to the semiconductor technology used in GPU chip manufacturing, measured in nanometers (nm) . The more advanced the process, the lower the power consumption and the higher the performance.

TDP (Thermal Design Power): A measure of the maximum power consumption of a GPU, which directly affects the cooling requirements. A higher TDP generally means a more powerful GPU, but it also means a better cooling system is needed.

Supported technologies : including NVlink technology, ray tracing, AI acceleration, virtual reality , etc. The support of these technologies not only shows the modernization of GPU, but also broadens its application areas.

So the question is, many friends in the IT field can understand the above content relatively easily, but many non-technical personnel in the intelligent computing center find it difficult to understand. In other words, how can "tech novices" quickly understand it? Next is the key point. We will introduce the various parameters of GPU cards to you by analogy.

3. Use analogies to quickly understand various GPU parameter specifications

Computing power: There are a lot of different computing power parameters, such as FP64, FP32, FP16, and FP16 Tensor Core. Which value should I use? Generally speaking, according to the current mainstream solution, 989TFPOPS of FP16 Tenser acceleration, about 1P, is usually used for calculation. Of course, some projects still use sparse computing power of 2P per card. Is there any reason for this?

In fact, this is mostly for marketing purposes. Generally, sparse matrix acceleration is used for numerical marketing, which has conditional restrictions on usage scenarios (not all scenarios can achieve the performance of sparse computing power).

Based on my personal understanding, let's take a not-so-appropriate example. We can compare GPU computing power to a truck, and different computing power values ​​to the running speeds that can be achieved by the truck in different gears. Tenser Core is like the speed that NVIDIA has optimized specifically for AI acceleration, just like the turbocharger system of a truck, which can provide the truck with more powerful performance.

Core Count : The core count is relatively easy to understand, and we can compare it to the number of cores in a CPU. Of course, the number of cores in a GPU far exceeds the number of cores in a CPU.

Here we can compare the number of GPU cores to a freight train with carriages. The more carriages a train has, the greater its ability to transport goods.

Core frequency: This can also be compared to the frequency of the CPU , which is equivalent to the transportation speed of the train. The higher the frequency, the faster the train transports goods.

Similarly, the higher the core frequency of a GPU, the faster it can process information.

Video memory type: The type of video memory used by the GPU directly affects its performance.

Common types include GDDR5, GDDR6, etc. Among them, GDDR6 provides higher speed and bandwidth than GDDR5. Different types of video memory are equivalent to carriages on different trains.

Video memory type: The size of the video memory determines how much data the GPU can "hold", an analogy to how much cargo each train car can hold.

The type of video memory is closely related to the bandwidth. HBM video memory is like the mechanical loading and unloading of goods on a train, which is fast and good, while ordinary GDDR is like the manual loading and unloading method, which is much slower.

Memory bandwidth:

It is equivalent to the maximum speed that a train can carry when unloading or loading goods. The higher the bandwidth, the faster the data exchange and the stronger the GPU performance.

Manufacturing process: This is equivalent to the process of making trains. The process of making ordinary green trains is definitely not as good as the process of making high-speed trains. The more advanced the process, the lower the power consumption and the higher the performance.

Supported technologies : This is like the more advanced trains support more functions. New technologies of GPU include ray tracing, AI acceleration, virtual reality, etc. The support of these technologies not only shows the modernization of GPU, but also broadens its application areas.

NVLink technology: Taking large model training as an example, the video memory capacity of a single card is limited and cannot hold the data for large model training. What should we do?

NVIDIA NVLink technology is used to solve the problem of multi-card collaboration . It is like if one carriage cannot accommodate all the items, multiple carriages need to be connected, which is equivalent to the connectors between each train carriage.

In short, with the rapid development of technology, GPU graphics cards have become an indispensable part of modern computer systems. When choosing a suitable GPU graphics card, it is crucial to understand its key parameters. Through the introduction of this article, I believe you have a deeper understanding of the key parameters of GPU graphics cards and can better choose a GPU that suits you.