✨
AI Summary
- GPU utilization is misleading; tensor core utilization shows true usage.
- Training at scale requires tracking tensor core utilization and InfiniBand counters.
- Most failures were cross-node communication, leading to custom solutions.
Guests on This Episode
GM
Gabriel Jorge Menezes
1 podcast appearance