✨
AI Summary
- FlashNorm folds RMS norm gain into projection weights to reduce computation.
- This technique aims to speed up transformers by optimizing the norm layer.
- It reduces arithmetic operations by deferring scalar division.
Guests on This Episode
FM
Filip Makraduli
1 podcast appearance