How AWS FSx for Lustre Boosts GPU Performance By 12x

AWS has announced that Amazon FSx for Lustre now supports the Elastic Fabric Adapter and NVIDIA GDS

[{"selector":"#anim-8a0ef997-73c5-4375-9b9b-4c2fe53b0e0e [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-e9941517-bc5a-46b5-8bf0-47674d1ad6e5","keyframes":{"transform":["translate3d(-115.55555%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b40cc293-6879-4e30-bbeb-fbe523878e55","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-de161537-3919-4293-bb38-1558655e5a9c","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Amazon FSx for Lustre with EFA/GDS support offers up to 12 times higher per-client throughput with these improvements

[{"selector":"#anim-a9e1e8a2-6509-42a8-8d27-01778f343724 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-38094bb8-7604-4836-b32b-0b5a9ac955b9","keyframes":{"transform":["translate3d(-115.55555%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-1d28ef7d-da34-4d49-a947-d353a5f06564","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-085d63df-9bfa-48e3-a021-686f2bd01df4","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

You can use more potent GPU and HPC instances, like Amazon EC2 P5, Trn1, and HPC7a, as datasets expand and new technologies appear

[{"selector":"#anim-dc6b08e8-702f-4e49-a8e5-c9b66b9740d2 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-e19b8f08-3fd7-4f4e-b9c6-d7d8347fe58a","keyframes":{"transform":["translate3d(115.23810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-15316bc0-1187-43bf-9ffd-0e01afe89bf9","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-13980dfd-ccc7-487a-bb50-a8a7f4f5960e","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

By removing superfluous memory copies and facilitating direct data transmission between the file system and GPU memory

[{"selector":"#anim-849367d9-8a30-4320-983b-ac4ce8fc283f [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-387056aa-5ac5-4dad-b9a2-823db50457b7","keyframes":{"transform":["translate3d(115.23810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-5fde8531-88d2-4c80-b32a-c35c863f693f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-45e3d3ba-5691-4185-a58b-3e3998f18031","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

You can start by selecting Create file system and then Amazon FSx for Lustre in the Amazon FSx panel

[{"selector":"#anim-4167ada6-0326-4fb2-8b15-9f6194de2f43 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-ae2ec3a2-dff9-4d50-89de-cc32134ed97b","keyframes":{"transform":["translate3d(-115.55555%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-016c8e7b-c934-482f-91a4-e64f2f02327f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-413d6919-3904-4749-98a5-6d226eaf4c8a","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

You can choose SSD, Persistent, and the new option with EFA enabled under the Deployment and Storage Type column

[{"selector":"#anim-16fdc2db-e532-4f74-9bdf-19df96bb337b [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-e73abf6a-af76-4a69-ad14-742e866395a7","keyframes":{"transform":["translate3d(115.23810%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-d0e56522-1498-4c71-a488-94ab49c1f8e2","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-704bd13b-d460-4c30-8813-ca9893eca216","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

In the Throughput per unit of storage area, You can choose 1000 MB/s/TiB

[{"selector":"#anim-848462ef-3165-4f24-995b-d02a0491f9ef [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]