Meta Releases Llama 3.1 405B Open Weights with 128K Context Window
Meta open-sourced Llama 3.1 405B, the largest open-weights language model rivaling proprietary frontier models in math, coding, and multilingual reasoning.
Verified State Diff
Comparison Mode:
- Previous State
Llama 3 context window was capped at 8K tokens without a flagship 400B+ frontier competitor.
+ Verified New State
128K context window with full FP8 weights available for self-hosting and model distillation.
Impact & Verification Analysis
WHO IS AFFECTED
AI researchers, open-source developers, and enterprises self-hosting private LLMs.
WHY IT MATTERS
Levels the playing field between closed proprietary frontier models and open-source AI infrastructure.
Full Fact Overview
Llama 3.1 405B was trained on over 15 trillion tokens using 16,000 H100 GPUs and includes an updated permissive license allowing developers to use model outputs for fine-tuning.
Multi-Source Evidence Chain (1)
TRACKED ENTITY
Explore all historical Meta changes