Gaudi 2 Intel Tests GenAI vs NV H100

Intel Gaudi 2 is Still the Only Benchmarked Option Besides NV H100

[{"selector":"#anim-e6eaba43-23fa-4044-a70c-7c7ad3c91d5c","keyframes":{"transform":["translate3d(151.50000%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-c524c846-90a1-4509-b6b9-5c2417edc5d3","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-bdd35aab-ac2a-4368-95ea-dffe4cb80ab6","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}] [{"selector":"#anim-00fe31d2-a2bb-43fa-8dc0-490e4ba8c347 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.249999904429615%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

The results of the industry-standard MLPerf v4.0 benchmark for inference were released by MLCommons

[{"selector":"#anim-d73716a6-0497-47fe-b48e-ee4753069a6b","keyframes":{"transform":["translate3d(152.51141%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4b287cf1-c174-4a4b-b5c2-dd9d27a2c48f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-9b3e5e8c-6c26-421c-8ded-3368a0faf4ef","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}] [{"selector":"#anim-67f280ff-9c69-460e-9e78-e107ac55bfdc [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.249999904429615%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

The Gaudi 2 Intel AI accelerator is still the sole benchmarked substitute for the Nvidia H100

[{"selector":"#anim-4db0cb26-9944-4e87-af3b-ce7769e0b7a9","keyframes":{"transform":["translate3d(137.21973%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-8ab8ed29-a425-4c79-9cdc-bf1de0ddb1d4","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d9795d6c-edfd-42f4-8514-ad933b128b65","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}] [{"selector":"#anim-07067a35-0644-4463-b989-33866d82b061 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.103260773104424%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

The average improvement in Intel’s 5th Gen Xeon results over 4th Gen Intel Xeon processors was 1.42x

[{"selector":"#anim-fc5e40c5-41e7-4532-9bb4-4a34e8e50126","keyframes":{"transform":["translate3d(152.99999%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-4af43f72-f070-49b9-b0da-133dcbc4bdd4","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}] [{"selector":"#anim-92155df5-29f6-41c1-9eb2-974f2573bbc2 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.44743368046207%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

This robust GPU is intended for usage in data centres for applications such as real-time deep learning inference

[{"selector":"#anim-b18702be-7396-4edf-9958-7d82c7a3fbc7","keyframes":{"transform":["translate3d(116.96114%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-2c75e8db-96cb-4038-b1b9-a49a8857eaec","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}] [{"selector":"#anim-33ccd061-1b71-4d2b-8b8b-211c9b45bd95 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-28.93339755556636%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

The H100 offers up to 7x better performance for HPC workloads than prior versions

[{"selector":"#anim-7851b7c0-8cdf-4834-93a5-8c9d4d15d029","keyframes":{"transform":["translate3d(150.49999%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5aeec5cd-9c3d-4959-b58c-815e7954e1cd","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-1a6aa62e-96c5-4e83-9563-17fafa092ab6","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}] [{"selector":"#anim-3a40b63a-d990-4dee-8b80-a75f56dd4019 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.249999904429615%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

The popular large language models (LLMs) and multimodal models that are covered by the Intel Gaudi software package are still growing

[{"selector":"#anim-0f2c9c19-76f1-4af6-a8e4-0f5e69e52944","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}] [{"selector":"#anim-882ce66e-b244-4c6f-97b9-7d371dd6d9c5 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.249999904429615%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

The Intel Developer Cloud offers evaluations of 5th generation Xeon CPUs and Intel Gaudi 2 accelerators for more details govindhtech.com

[{"selector":"#anim-5ac88796-3f64-409d-acbc-c7538da5292c","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}] [{"selector":"#anim-8d51bc98-2ed8-496e-b0ce-7e522fccc5b1 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-33.84712828036338%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]