Episodes (Page 7)
✨
A subtle bug in vLLM caused gibberish output for a specific model.
Asaf Gardin
Yuval Belfer
✨
Agents are changing the economics of AI inference, according to FriendliAI.
Byung-Gon (Gon) Chun
✨
Small models on older GPUs offer a cost-effective alternative for embeddings.
Daniel Svonava
✨
OpenAI's inference load balancer uses a feedback loop for routing weights.
Lu Zhang
Qianru Lao
✨
Improvisation funded GitHub Codespaces for a workshop.
✨
TCP and RDMA are bottlenecks for modern AI workloads.
✨
Agent token usage is heavily spent on upfront search.
Maximilian-David Rumpf
✨
Most websites have llms.txt, but agents ignore it.
✨
Exa initially didn't plan an API for their search engine.
Will Bryk
✨
BM25, a 30-year-old lexical function, is effective for agents.
Jo Kristian Bergum
✨
Small text chunks fail for answers spread across scenes.
Yuval Belfer
✨
Search latency spiked due to a packing problem.
✨
Companies cannot easily query negotiated contract values.
✨
Agents struggle to differentiate context in text.
Benjamin Clavie
0:19:50
✨
Real-world voice agents face strict latency constraints, unlike demos.
Venky B
✨
Voice assistants focus too much on accuracy, ignoring user experience.
✨
Voice agents struggle with non-standard pronunciations and user corrections.
Midam Kim
✨
Bohan Li uses background agents to mask latency in slow models.
Frontier Intelligence
✨
Sumanyu Sharma's voice agent error cost him a doctor's appointment.
Sumanyu Sharma