✨
AI Summary
- MosaicML's MPT-7B achieved SOTA in open-source with 84,000 token context length (vs GPT-3's 4,000), trained on 1 trillion tokens matching LLaMA-7B quality
- Extended context enables new applications without requiring retrieval augmentation, representing breakthrough for commercially licensed open models
- Context-as-infinity paradigm shift moving from limited token windows to practical document-length processing within single model pass