Lec 10 - Prefill-Decode Disaggregation Past, Present, and Future

Batching Prefill and Decode Together Cause Interference

Resource and Parallelism Coupling

Challenges

1. Prefill and decode interference
2. Resource and Parallelism Coupling

Solution

• Put prefill and decode on different workers

DistServe - Parallelism Strategy