500AI
Back to researchers

Tri Dao

Efficient sequence models + attention kernels

Co-authored Mamba and FlashAttention/FlashAttention-2: high-leverage work for fast training and inference.

Highlights

MambaFlashAttentionSystemsEfficient sequence models
Focus: Efficient sequence models + attention kernels
Why it matters: Co-authored Mamba and FlashAttention/FlashAttention-2: high-leverage work for fast training and inference.
Start here
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesspaper →FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningpaper →Mamba: Linear-Time Sequence Modeling with Selective State Spacespaper →

Research Areas

MambaFlashAttentionSystemsEfficient sequence models
Tri Dao - AI Researcher Profile | 500AI