You will not discover Nvidia’s H100 (Hopper) GPU on the listing of the finest graphics playing cards. Nonetheless, the H100’s forte lies in synthetic intelligence (AI), making it a coveted GPU within the AI trade. And now that everybody is leaping on the AI bandwagon, Nvidia’s H100 has change into much more well-liked.
Nvidia claims that the H100 delivers as much as 9X quicker AI coaching efficiency and as much as 30X speedier inference efficiency than the earlier A100 (Ampere). With a efficiency of that degree, it is simple to grasp why everybody desires to get their fingers on an H100. As well as, Reuters (opens in new tab) reported that Nvidia had modified the H100 to adjust to export guidelines in order that the chipmaker may promote the altered H100 because the H800 to China.
Final 12 months, U.S. officers carried out a number of laws to stop Nvidia from promoting its A100 and H100 GPUs to Chinese language purchasers. The foundations restricted GPU exports with chip-to-chip information switch charges beneath 600 GBps. Switch pace is primordial within the AI world, the place techniques have to maneuver monumental quantities of knowledge round to coach the AI fashions, resembling ChatGPT. Hindering the chip-to-chip information switch charge ends in a big efficiency hit, because the slower switch charges improve the time it takes to switch information, in flip rising the coaching time.
With the A100, Nvidia trimmed the GPU’s 600 GBps interconnect right down to 400 GBps and rebranded it because the A800 to commercialize it within the Chinese language market. Nvidia is taking an equivalent method to the H100.
In keeping with Reuters’ Chinese language chip trade supply, Nvidia decreased the chip-to-chip information switch charge on the H800 to roughly half of the H100. That would depart the H800 with an interconnect restricted to 300 GBps. That is a extra vital efficiency hit than in comparison with the A100 and A800, the place the latter suffered from a 33% decrease chip-to-chip information switch charge. Nonetheless, the H100 is considerably quicker than the A100, which may very well be why Nvidia imposed a extra extreme chip-to-chip information switch charge restrict on the previous.
Reuters contacted an Nvidia spokesperson to inquire about what differentiates the H800 from the H100. Nonetheless, the Nvidia consultant solely acknowledged that “our 800-series merchandise are absolutely compliant with export management laws.”
Nvidia already has three of probably the most outstanding Chinese language expertise corporations utilizing the H800: Alibaba Group Holding, Baidu Inc, and Tencent Holdings. China has banned ChatGPT; subsequently, the tech giants are competing with one another to provide a home ChatGPT-like mannequin for the Chinese language market. And whereas an H800 with half the chip-to-chip switch charge will undoubtedly be slower than the full-fat H100, it can nonetheless not be sluggish. With corporations doubtlessly utilizing hundreds of Hopper GPUs, finally, we’ve to marvel if this may imply utilizing extra H800s to perform the identical work as fewer H100s.