10 interesting facts about the NVIDIA Hopper H100 GPU

At the Graphics Technology Conference (GTC), NVIDIA introduced the latest AI accelerator – the H100 Tensor Core GPU. It is a successor to the hugely successful A100 GPU that was launched two years ago. With 9x faster AI training and 30x faster inference, it’s more than just an incremental upgrade to A100.

Here are ten facts about NVIDIA’s next-gen GPU:

1) The GPU is based on the Hopper architecture, a successor to the Ampere architecture that powers A100 and A30 GPUs. The A100 GPUs are available through the NVIDIA DGX A100 and EGX A100 platforms.

2) Compared to A100 GPUs which support 6912 CUDA cores, H100 has 16896 CUDA cores. NVIDIA GPUs have CUDA cores, which correspond to CPU cores. You can run many calculations simultaneously, which is essential for modern AI and graphics workloads.

3) The H100 GPU is packed with 80 billion transistors, while the previous generation of A100-based GPUs had 54.2 billion transistors. This increase in transistors results in faster calculations and processing.

4) The H100 GPU adopts 2nd generation secure multi-instance GPU (MIG) with 7 times expanded functions over the previous version. NVIDIA claims that the new GPU architecture offers about 3x more compute capacity and almost 2x more memory bandwidth per GPU instance than A100.

5) Sensitive computing support in H100 protects user data, protects against hardware and software attacks, and provides better isolation and protection for virtual machines (VMs) from each other in virtualized and MIG environments.

6) The H100 GPUs feature 4th Gen NVLink, which provides a 3x bandwidth increase on All Reduce operations and an overall 50% bandwidth increase over the previous generation NVLink. NVIDIA NVLink is a direct GPU-to-GPU connection that scales multi-GPU input/output (IO) within the server.

7) The H100 GPU has built-in support for DPX instructions, which speeds up dynamic programming algorithms by up to 7X over the A100 GPU. Dynamic programming was developed in the 1950s to solve complex problems using two key techniques based on recursion and memoization. Applications based on complex SQL queries, quantum simulation and route optimization can take advantage of the DPX command set available in H100.

8) The H100 GPU is optimized for Transformers. The built-in Transformer engine uses a combination of software and custom NVIDIA Hopper Tensor Core technology explicitly designed to accelerate Transformer model training and inference. Transformers represent the latest developments in neural network architecture for training computer vision and conversational AI models. They are used in major language models such as Google’s BERT and OpenAI’s GPT-3.

9) NVIDIA updates HGX AI supercomputing platform with H100 GPUs. The HGX platform allows hardware vendors to design servers that are optimized for NVIDIA GPUs. The availability schedule for the HGX platform based on H100 GPUs has yet to be announced.

10) Systems based on H100 GPUs will be available in Q3 2022. This includes NVIDIA’s own DGX and DGX SuperPod servers, as well as servers and hardware from OEM partners that use HGX baseboards and PCIe cards.

Leave a Reply

Your email address will not be published.