Advanced strategies for rapid, accurate, and efficient on-device generative AI models

With generative AI use rising at historic rates and compute needs rising, on-device AI processing is crucial The first Android phone-based Stable Diffusion demo was shown at MWC 2023

[{"selector":"#anim-a9869f14-1365-4227-874d-9897439a8bb3 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-58ff5596-3b06-464e-bfb3-34e32f7288e9","keyframes":{"transform":["scale(1)","scale(1.5)","scale(0.95)","scale(1)"],"offset":[0,0.33,0.66,1]},"delay":0,"duration":1450,"easing":"ease-in-out","fill":"both","iterations":1}]

After Snapdragon Summit 2023, we exhibited some stunning On-Device AI demonstrations on Snapdragon 8 Gen 3 Mobile Platform smartphones and Snapdragon X Elite Platform computers

[{"selector":"#anim-055a5e57-4bc0-42ea-b923-a0664be7e8e0 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(7.105427357601002e-15%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-f0f88e16-bc0d-4599-aa0d-9902e68a98b8","keyframes":{"opacity":[1,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-0052f6aa-13e1-457a-a7ba-1097c1bb4408","keyframes":{"transform":["scale(3)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

They keep optimizing our solutions using full-stack On-Device AI and pushing edge technology

[{"selector":"#anim-a67ec93e-1288-4d1d-a4f2-9fa0fda4df96 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.179687404002955%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-a4f2a9e0-b091-4e87-a64b-327c850326fc","keyframes":{"transform":["translate3d(153.30190%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-38ad41c3-4236-46ea-9d8d-998f358bed37","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

Knowledge distillation is a transfer learning strategy that trains a smaller “student” model to replicate a bigger “teacher” model with high accuracy

[{"selector":"#anim-f45be74b-6c23-4880-b738-e47ff05321ae [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

They developed basic methods to decrease quantization errors, many of which are available in the AI Model Efficiency Toolkit (AIMET) on the Qualcomm AI Stack and GitHub

[{"selector":"#anim-11e05a7b-a706-4c76-8f28-b104fa0f59de [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(34.179687404002955%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-66b89b47-a180-4cd7-a6fe-58abb7f7fccd","keyframes":{"transform":["translate3d(128.73564%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d9796cf3-1fdb-4070-8b26-f329e174f19c","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-12d4dbf4-d8cc-43a5-9f30-dbab271bc44d","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Our Stable Diffusion implementation ran in under a second at Snapdragon Summit We accomplished it how? As its name says, Stable Diffusion creates visuals using reverse diffusion conditioned on the input prompt

[{"selector":"#anim-7663b69c-cf12-4755-a62f-393f3ebe05ab [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-32.667617583843466%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-59915a5d-811e-4fd8-88ee-14287c09092a","keyframes":{"transform":["translate3d(115.23810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-fb84d65b-048f-4b22-84a7-94515b86feba","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-28dcfa89-e938-47a7-a0c8-a53087e8b6f6","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

We created the quickest phone diffusion-based text-to-image generation by combining these methods Running on a Snapdragon 8 Gen 3 phone, we got a 9x speedup against the baseline Stable Diffusion and over 25x faster than at MWC 2023

[{"selector":"#anim-e5efc261-45a0-416a-83cf-eab0b748958f [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(34.179687404002955%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}]

Effective On-Device AI via knowledge distillation and quantization-aware training For more details visit govindhtech.com

[{"selector":"#anim-c0d52ba1-8f9a-416a-a70d-80e5fba65513 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(29.756042849407542%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-d9df6b5e-36ec-4acc-a4c2-fadb16fa3441","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-fc3c27bb-1d2e-47d0-955f-7c03e90385e3","keyframes":{"transform":["translate3d(0px, 156.82287%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]