✨
AI Summary
- Attention Residuals (AttnRes) transform LLM information aggregation across layers.
- Sequence-Depth Duality explores applying linear to softmax attention to model depth.
- Fixed unit weights cause uncontrolled hidden-state growth and diluted layer contributions.