✨
AI Summary
- Speculative decoding uses a small model to speed up large model inference.
- Evaluate trade-offs: faster inference vs. memory costs of two models.
- Choose draft models carefully to optimize performance gains.
Guests on This Episode
SK
Sheilah Kirui
1 podcast appearance