NVIDIA Isaac Sim

NVIDIA and university experts are working together to prepare robots for surgery

[{"selector":"#anim-63c56df1-e68c-435e-af33-912e725551c4 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(37.12349389776948%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-5bf8dc0b-4c6f-4839-ba19-c0679cf3381e","keyframes":[{"offset":0,"transform":"translate3d(0, -178.84367%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.29,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.45,"transform":"translate3d(0, -50.290840004%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.61,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.71,"transform":"translate3d(0, -17.097454852000002%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.8,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.85,"transform":"translate3d(0, -6.420487753000001%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":0.92,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"},{"offset":0.96,"transform":"translate3d(0, -2.789961252%, 0)","easing":"cubic-bezier(.5, 0, 1, 1)"},{"offset":1,"transform":"translate3d(0, 0%, 0)","easing":"cubic-bezier(0, 0, .5, 1)"}],"delay":0,"duration":600,"fill":"both"}]

NVIDIA Isaac Sim, a robotics simulation platform for creating, honing, and testing AI-based robots, was used to build the physics-based framework

[{"selector":"#anim-8fdb827e-3705-4cbb-a3b4-2fd811c86870 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(31.284090795523493%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-ef3368e7-b440-469f-aa4c-8d22202b849f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-f3f75d91-9e62-47b4-bbcc-24395860878a","keyframes":{"transform":["translate3d(0px, 122.48741%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]

The foundation of ORBIT-Surgical is Isaac Orbit, a modular robot learning framework based on Isaac Sim

[{"selector":"#anim-279716ce-93eb-4605-8113-d41effdeb001 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-6e9f9c6a-2bdc-48a5-9ab4-7925518e4b78","keyframes":{"transform":["translate3d(115.84158%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-e56c0918-576b-4fba-83e3-b054a201eca6","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

With the surgical framework, developers may use reinforcement learning and imitation learning frameworks running on NVIDIA RTX GPUs to train robots

[{"selector":"#anim-a8eee2e2-b462-4a47-97c0-63ccbefd897e [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.249999904429615%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-c350c29d-4384-47b6-8d0c-65d42321466a","keyframes":{"transform":["translate3d(121.73912%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-55e4a388-3267-43c9-863c-cda2307973f5","keyframes":{"transform":["rotateZ(180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

Many will be talking about geometric fabrics at the IEEE International Conference on Robotics and Automation

[{"selector":"#anim-13443f51-8216-41b9-ad22-e7076d608a1f [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-32.96929055409776%, 0, 0) translate(-25%, 0%) scale(1.5)","translate3d(0%, 0, 0) translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"fill":"forwards"}] [{"selector":"#anim-5d366737-9bc3-403d-8be7-20267ff2ff38","keyframes":{"transform":["translate3d(-117.37705%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .8, 1)","fill":"both"}] [{"selector":"#anim-2121c766-a0b3-495b-b614-eeb794b79bd2","keyframes":{"transform":["rotateZ(-180deg)","rotateZ(0deg)"]},"delay":0,"duration":1000,"easing":"cubic-bezier(.2, 0, .5, 1)","fill":"forwards"}]

One of the seven publications that members of the NVIDIA Robotics Research Lab and their collaborators have submitted addresses that topic

[{"selector":"#anim-cbcfea1d-bcda-4e40-b901-fdc8499ed147 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-34.249999904429615%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-2c78c054-e709-4ffe-b2e5-4bf406bcfa4a","keyframes":{"transform":["rotate(-540deg) scale(0.1)","none"],"opacity":[0,1]},"delay":0,"duration":1000,"fill":"both","iterations":1}]

NVIDIA ORBIT-Surgical, an NVIDIA Omniverse-based surgical robot simulation system, uses Isaac Sim for photorealistic rendering

[{"selector":"#anim-c71241d0-ae4f-4b29-bc81-4a5c6b8639be [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(27.877964292641817%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-1a105ea2-85ed-49ac-bd43-e3a52eba49c8","keyframes":{"transform":["translate3d(-121.73913%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ce967909-7e46-4228-8aa8-066d80cf294f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-c7b9120d-3a84-456b-a53c-04842a8d1614","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Munich-based Franka Robotics showcases their work using the Franka toolkit for Matlab and NVIDIA Isaac Manipulator For More Details Govindhtech.com

[{"selector":"#anim-cad0e1b0-30cb-4db7-aa3d-de951fdca858 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-28.93339755556636%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-23df0072-73ac-4318-b1bd-a1e29f2d8e97","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}] [{"selector":"#anim-619e5824-8383-428b-b728-1398adae6937","keyframes":{"transform":["translate3d(0px, 152.58576%, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.2, 0.6, 0.0, 1)","fill":"both"}]