500AI
Back to researchers

Rohan Varma

Fully Sharded Data Parallel training (FSDP)

Co-authored PyTorch FSDP: practical lessons for scaling fully-sharded training workloads.

Highlights

SystemsTrainingPyTorchScaling
Focus: Fully Sharded Data Parallel training (FSDP)
Why it matters: Co-authored PyTorch FSDP: practical lessons for scaling fully-sharded training workloads.
Start here
PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelpaper →PyTorch Distributed (docs)website →

Research Areas

SystemsTrainingPyTorchScaling
Rohan Varma - AI Researcher Profile | 500AI