AMD ROCm 6.2

Introducing the Newest AMD ROCm 6.2 Release, Unleashing Next-Gen AI & HPC Performance

[{"selector":"#anim-4c7335fc-2e75-469c-afd8-1488d18d0ea1 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.13461528834485%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

AMD is adding vLLM support to AMD Instinct Accelerators to increase AI model performance and scalability

ROCm/vLLM offers FP8 GEMMs with bespoke decode paged attention for cutting-edge performance

[{"selector":"#anim-566836f4-5216-4ef3-b809-0984cdb94f35 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-32.716480342051646%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

New and existing AMD Instinct customers may simply integrate vLLM into their AI pipelines with ROCm 6.2

AI development is revolutionized by the Bits and bytes quantization library support via AMD ROCm

[{"selector":"#anim-80eb335c-fcb4-4b5a-83db-6cd5b671c381 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(32.716480342051646%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}]

AI is optimized via LLM.Int8() quantization, enabling efficient LLM deployment on devices with lower memory

Bits and bytes democratizes AI development, gives cost savings, increases innovation potential, and makes advanced AI capabilities

[{"selector":"#anim-a82e2f8d-3049-4ec5-b25b-8de62e2db31f [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.13461528834485%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}]

Wide FP8 Support in ROCm can greatly enhance the AI model execution process, especially for inferencing

[{"selector":"#anim-bd465e81-3d13-4905-b6ee-bba7af7019db [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.21556112871043%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}]

FP8’s lower precision calculations can reduce the latency associated with computations and data transfers

ROCm 6.2 supports FP8 in its frameworks, libraries, and more to improve performance and efficiency

[{"selector":"#anim-6ced30b5-b32c-4f70-a633-515bc1159aa0 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-44.42246832058613%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-05d81006-e1b6-4db3-90e5-a4f8673df25f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}]