✨
AI Summary
- Agentic requests have high input token reuse, making prefill expensive.
- Caching input tokens is crucial for efficient inference stack design.
- CoreWeave's platform serves both serverless and provisioned inference models.
Guests on This Episode
SG
Sitanshu Gupta
1 podcast appearance