Large Multimodal Models(LMM)

Despite its unlimited capacity, generative artificial intelligence (AI) can only do so much due to its environmental perception

[{"selector":"#anim-25d96447-2724-4d56-bdd2-a7feb7f36f15 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-35.34967836769406%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-99508965-bc32-49af-b2b0-b738ba158f4d","keyframes":{"transform":["translate3d(-118.21191%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-bc13b97f-adfd-46f0-b99e-e60886f8e9ce","keyframes":{"transform":["rotateZ(-180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

Generative AI research after the commonly used Large Language Models (LLMs), including the ChatGPT original model, which could only parse text

[{"selector":"#anim-06eaa7d0-5e32-4e6d-a319-cf34ad874e20 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(35.34967836769406%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-c5229c03-d67f-4e3c-9a3f-6b45eef3b978","keyframes":{"transform":["translate3d(-111.63522%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-2421afae-4c36-4b86-8416-184063cf5ae6","keyframes":{"transform":["rotateZ(-180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

Digital assistants and productivity tools will also become much more helpful

[{"selector":"#anim-12b7c296-18a5-431d-8e03-bd04603d03bf [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-21.874999829338588%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-50fb82a3-1561-4b5f-9a81-18be968ea911","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

Qualcomm Technologies strives to enable multimodal AI on devices

[{"selector":"#anim-4a88a3d1-2fb9-4966-8cd8-ef50144cfa83 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-2039e889-bc15-4233-a773-ff55f0e4f2f1","keyframes":{"transform":["translate3d(-162.99999%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-efa0591a-89b3-4340-9cf7-1b9810505a22","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-46693ff6-fbd6-4077-9a54-3e0126edb24d","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

LLaVA, a community-driven LMM with over seven billion parameters, on a Snapdragon 8 Gen 3 Mobile Platform-powered Android phone

[{"selector":"#anim-caadc221-e125-4960-bec7-665083b60912 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-59b75760-a5fd-4e3b-9a47-acb4f0538a1d","keyframes":{"transform":["translate3d(115.18987%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-7b24a86c-c3f4-4847-aadd-ca9806932e4c","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3c2ded04-d54d-4aac-b65d-588895d6728b","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Given the multimodal hype, this work is vital. Last week, Microsoft introduced the Phi-3.5 family of visual and linguistic devices

[{"selector":"#anim-7db17de0-f2ee-4a43-ac5d-c981c8b63a47 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-35.21169345865223%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-e0971f91-3d54-4e48-96cf-78a2c2947775","keyframes":{"transform":["translate3d(-115.18988%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-3d2e8d5d-11eb-4a5b-bc9f-505c595ed36e","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-31233a93-0995-4c22-9163-d8ce59c18ab6","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

At its Made by Google presentation, Google promoted LMMs and introduced the multimodal input model Gemini Nano

[{"selector":"#anim-b3fa0a99-43e0-4605-839b-b4028e1ee0d9 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(34.39576402491543%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-e0e88a12-ea91-4dd9-80f5-12f392a1b997","keyframes":{"opacity":[1,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-8b0a0990-f7dc-469c-8ab0-b3911f9bd21e","keyframes":{"transform":["scale(3)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Qualcomm Technologies is partnering with Google and many LMM and LLM manufacturers, including Meta's Llama series

[{"selector":"#anim-e508638d-2154-4a26-a447-335c2a8fbacf [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-21.874999829338588%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-3273eebe-e3d6-4531-9524-5a909cc12b44","keyframes":{"transform":["translate3d(122.22222%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-ee4a743e-af7c-4c1c-ad3c-ede86a5a81a8","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

Multimodal AI can use cameras, microphones, and automotive sensors to identify bored backseat passengers and provide amusement For more details Govindhtech.com

[{"selector":"#anim-8c8bbf10-112c-4f8a-b39b-027b2b06d391 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(31.284090795523493%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-ccd08049-7508-4160-ad78-2fbe718068e0","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]