Skip to content
Sign in
Artwork for PyTorch Developer Podcast
PyTorch Developer Podcast · Feb 19, 2024 · 18 min

Compiled autograd

Compiled autograd is an extension to PT2 that permits compiling the entirety of a backward() call in PyTorch. This allows us to fuse accumulate grad nodes as well as trace through arbitrarily complicated Python backward hooks. Compiled autograd is an important part of our plans for compiled DDP/FSDP as well as for whole-graph compilation.

0:00-18:07
Play in

transcript

No transcript — this publisher did not publish one.

show notes

Compiled autograd is an extension to PT2 that permits compiling the entirety of a backward() call in PyTorch. This allows us to fuse accumulate grad nodes as well as trace through arbitrarily complicated Python backward hooks. Compiled autograd is an important part of our plans for compiled DDP/FSDP as well as for whole-graph compilation.

more episodes

All episodes