MaxDiffusion: Diffusion model inference efficiency

There is a growing need for high-performance, low-cost AI inference (serving) in the quickly changing field of artificial intelligence

[{"selector":"#anim-10c6f522-1995-465d-a12b-84313d234ad0 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-975213bc-ac77-4155-aa57-816b872f659d","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

JetStream is particularly designed for LLMs and marks a major advancement in both performance and cost effectiveness

[{"selector":"#anim-747d0501-4737-4236-b13f-645093f85010 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-16a5701e-80a6-483c-a158-bbbee2be9a6a","keyframes":{"transform":["translate3d(126.29630%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-dbd678a8-48ea-494e-9da5-307c424df811","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-9cc1bf60-332c-415a-9586-fedd015fc0b0","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Google is pleased to provide the most recent MLPerf Inference v4.0 performance results

[{"selector":"#anim-67b845c5-85ee-4df0-97ee-39a05d228fb0 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(-7.105427357601002e-15%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-1e24631a-8fbc-4989-9bab-ddb13e39fe95","keyframes":{"transform":["translate3d(138.79310%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-902685ca-d99b-46e5-9ed4-79cc383c6c2a","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-03957d5f-f1ab-4b43-ab57-58e68834dac7","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Google developed JetStream, an inference engine that offers up to three times more inferences per dollar than earlier Cloud TPU inference engines

[{"selector":"#anim-45f11927-918b-43ff-aef8-32db59b3d606 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(29.20258608069865%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-6b4b5044-338c-4675-934c-3e756ea960cf","keyframes":{"transform":["translate3d(124.01433%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-0e0e8a9f-4fea-4295-940e-4b9599f5f8b0","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

JetStream supports your favourite framework, whether you’re using PyTorch or JAX

[{"selector":"#anim-325637b4-e042-4a62-a1d3-1d5a7daa22a2 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(36.458333251163026%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-63b14b34-2256-452d-92c6-4787fe07e894","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

JetStream provides up to 4783 tokens/second for open models, such as Gemma in MaxText and Llama 2

[{"selector":"#anim-823358d4-469f-4c63-b652-c8b7e8c833b1 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-29.20258608069865%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-eb47ee9a-a44d-482f-8f98-a65fecfa5db8","keyframes":{"transform":["scale(1)","scale(1.5)","scale(0.95)","scale(1)"],"offset":[0,0.33,0.66,1]},"delay":0,"duration":1450,"easing":"ease-in-out","fill":"both","iterations":1}]

Google is decided on Cloud TPU v5e with MaxText, JAX, and JetStream for this reason

[{"selector":"#anim-489fb827-2396-4197-9a85-afd0a17ab0cf [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-a65eb7d8-7df0-4249-8cad-b754967aaf63","keyframes":{"transform":["translate3d(-142.5%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-40494f05-b53c-4b66-ae5d-b60ee89394b2","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-54985b0c-735b-4e57-9598-054c07cad901","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Google’s AI inference innovation enables their clients to develop and expand AI applications for more details govindhtech.com

[{"selector":"#anim-3be21b79-e246-4277-95e6-7bcd7829bc16 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(37.4441963523833%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-4d30c814-c3b2-4985-ab3b-2b07c3f7319b","keyframes":[{"offset":0,"transform":"translate3d(0, -151.71958%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.29,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.45,"transform":"translate3d(0, -42.663545896%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.61,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.71,"transform":"translate3d(0, -14.504391848000001%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.8,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.85,"transform":"translate3d(0, -5.446732922000001%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.92,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.96,"transform":"translate3d(0, -2.366825448%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":1,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"}],"delay":0,"duration":600,"fill":"both"}]