Mistral previews Large 4, a 1-trillion-parameter open-weight model
Mistral's post makes the case for open weights in security work, where a provider's refusal can stall incident response.
Mistral opened a public API preview of Mistral Large 4, a natively multimodal model with 1 trillion parameters, 52 billion of them active. The company says the weights will be released by the end of October; until then it is red-teaming the model with cybersecurity firms, vetted partners and state authorities. Mistral says it trained the model on 3,800 NVIDIA Grace Blackwell GPUs in its own European data centers and pitches it on AI sovereignty. Its performance comparisons are the company's own.
-
Mixture of experts
After exploring Introducing Mistral Large 4, the technique behind "52 billion active" is Mixture of experts.
A machine-learning technique that splits a problem among multiple expert networks, each handling its own region of the problem space. That's why only part of a large model runs for any given input.
Wikipedia · Swim · 30 min
0