PyTorch/XLA 2.5: vLLM Support And Developer Improvements

PyTorch/XLA, a Python package that connects the PyTorch deep learning framework with Cloud TPUs via the XLA deep learning compiler

[{"selector":"#anim-4cd074ce-06e5-4449-9f0b-6071aafeaed4 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-792b3870-ab2f-41a6-a135-40020bb8c7f4","keyframes":{"opacity":[1,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-f4d23ba7-29cf-418a-9f6a-242351e9c95c","keyframes":{"transform":["scale(3)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Additionally, PyTorch/XLA 2.5 has arrived with a number of enhancements to improve the developer experience and add support for vLLM

[{"selector":"#anim-82fef601-f00a-4908-9f71-6650fc2868e7 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-32efcee3-726b-4675-a1f5-06644781fd5a","keyframes":{"opacity":[1,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-cca8a832-c427-429d-92c5-fb07a7881cc7","keyframes":{"transform":["scale(3)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

Google Cloud is making a big stride toward improving the consistency of the API with upstream PyTorch with PyTorch/XLA 2.5

[{"selector":"#anim-15384ed7-1edb-414c-94f3-b3cebc1fc8eb [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-d5a1ea02-9434-48e2-bf2a-28f0b8973d3e","keyframes":{"transform":["translate3d(-116.21622%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-4d144691-022e-4251-8aa6-71d14ff23e28","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-945e4459-dfae-44b9-b7b3-50cca597c328","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

PyTorch/XLA API calls in favor of more sophisticated functionality, then switching the API calls to their PyTorch equivalents

[{"selector":"#anim-a413117b-c4cd-49bf-b5f3-c5c75c9d2b2f [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-f353f4a1-c8ec-43e9-8797-fd4b9fedf153","keyframes":{"transform":["translate3d(-115.55555%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-681fe755-d433-4588-9555-821159fd9548","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-011de4af-f75d-45e2-b9c0-362b014c914d","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

Its goal is to make XLA devices easier to use by reducing the learning curve for developers who are already familiar with PyTorch

[{"selector":"#anim-431faa87-2433-4515-8041-8028208f82ee [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}] [{"selector":"#anim-e7767234-4db2-4898-858c-2042980403ec","keyframes":{"transform":["translate3d(-120.13202%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-ad1edca4-e346-4432-8dc4-ed0ce7db5c84","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-0bfabca3-0618-4d70-8294-e743e6e20ec7","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

In this release, it moved the majority of the calls for the distributed API from the torch_xla module to torch

[{"selector":"#anim-db54b03f-7573-4ca6-b260-dc7b44044a92 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(-12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-711f6e98-b1ee-460d-9e31-ab3a17dc62d8","keyframes":{"opacity":[1,1]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-5175965d-a9dd-4c5c-a8f5-b8bf0e837371","keyframes":{"transform":["scale(3)","scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]

For installation instructions and more thorough information, see the project’s GitHub page if you’ve never heard of PyTorch/XLA before

[{"selector":"#anim-dabf23b8-efa1-4133-bf67-e4d589caa664 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate3d(12.514648210040342%, 0, 0)","translate3d(0%, 0, 0)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"both"}] [{"selector":"#anim-a1dec0a2-e726-4346-a206-80f79ef7438e","keyframes":{"transform":["translate3d(113.75000%, 0px, 0)","translate3d(0px, 0px, 0)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-49ce3f9f-289b-4ce7-958e-828d58c07f7f","keyframes":{"opacity":[0,1]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"both"}] [{"selector":"#anim-b37973b4-9a2f-48c9-9bca-db49704b4947","keyframes":{"transform":["scale(0.15)","scale(1)"]},"delay":0,"duration":600,"easing":"cubic-bezier(0.4, 0.4, 0.0, 1)","fill":"forwards"}]

For more details visit Govindhtech.com

[{"selector":"#anim-07f76a4c-24b5-464d-b437-6e78843143c1 [data-leaf-element=\"true\"]","keyframes":{"transform":["translate(0%, 0%) scale(1.5)","translate(0%, 0%) scale(1)"]},"delay":0,"duration":2000,"easing":"cubic-bezier(.3,0,.55,1)","fill":"forwards"}]