NVIDIA Fugatto AI: Creates Music, Voices & Sounds On Demand

A new generative AI model Fugatto from NVIDIA can produce any mix of music, speech, and noises given text and audio as inputs

[{"selector":"#anim-b0f7a8b6-428f-4f4a-afb2-326593685d23 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-aa3636ff-0aaa-4d3a-97e0-392eaaaa7933","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-52e0235e-b0e4-4c9d-9edd-42ea1437ce36","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Music producers may use Fugatto to rapidly modify or prototype a song concept, experimenting with various instruments, vocals, and genres

[{"selector":"#anim-05c05dc0-b4d7-4ee1-8e55-7a9fb8c2b702 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-8dc2d7b3-c56c-44c3-8bef-9302ee87f05a","keyframes":{"transform":["translate3d(115.23810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-30827bfe-ade6-425b-b395-acd3c77bb3d2","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-70c4b724-2195-4731-a5a1-85d94d66506a","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

By applying various dialects and emotions to voiceovers, an advertising firm may use Fugatto to swiftly target an existing campaign

[{"selector":"#anim-7637c2ff-2de0-4b1f-b259-141483e7aa07 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-d57d6c66-cb61-4282-be89-dd3c5dd20bce","keyframes":{"transform":["translate3d(-115.55555%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-66545cce-478e-4121-bb8c-32005ee6185b","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-cba84a87-71d6-4440-90ff-bd57cd735db8","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

For example, Fugatto can meow on a saxophone or bark on a trumpet. The model can generate whatever that users may describe

[{"selector":"#anim-c6a2f71d-b6af-4030-ab8e-75e06bd45449 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-83f98b29-d50f-408f-aba9-b691bc1ab695","keyframes":{"transform":["translate3d(115.23810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-b03cf19b-54b8-4af2-a99a-3d2abe71e93d","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

Fugatto lets users construct soundscapes they have never seen before, such a thunderstorm fading into a morning with the sound of birdsong

[{"selector":"#anim-4c6507fa-a37e-409f-99a1-6fc67acc3203 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-91eb0c2a-9976-45fe-9fc1-b3f8f6e255be","keyframes":{"transform":["translate3d(115.23810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-80f5ee39-3502-45ee-b8a6-33eda99c0d81","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

The entire version was trained on a bank of NVIDIA DGX computers with 32 NVIDIA H100 Tensor Core GPUs and employs 2.5 billion parameters

[{"selector":"#anim-16cb716d-5c58-4804-a314-22f98316d6b7 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-193fc5d5-3a80-4b5f-81f9-a199ffaf5d97","keyframes":{"transform":["translate3d(115.55555%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-03582a1c-bdc3-4f6d-a0d0-021c9494940d","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ba3762b9-7648-4308-b3e3-f698d102faa9","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Additionally, they examined pre-existing datasets to uncover novel connections between them. The entire project took almost a year to complete

[{"selector":"#anim-028fef9a-1979-4a39-b393-b472dc7b9ad7 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-e0cce5e8-3289-4494-8a7a-653ee67640ab","keyframes":{"opacity":[0,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-9b5107a5-f329-4121-8e0b-cbd9e62373a9","keyframes":{"transform":["scale(0.3333333333333333)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

For more details visit Govindhtech.com

[{"selector":"#anim-b1233e53-9b5f-432d-b4ef-f4b755e01590 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}]