Leanstral 1.5: Proof Abundance for All
Mistral AI releases Leanstral 1.5, an open-source Lean 4 model with 6B active parameters, improving formal verification performance on benchmarks like miniF2F and PutnamBench while uncovering real-world bugs in open-source repositories.
Mistral AI has introduced Leanstral 1.5, a free Apache-2.0 licensed model with 6B active parameters out of 119B total, designed to enhance formal verification in Lean 4. The model achieves state-of-the-art results on FATE-H (87%) and FATE-X (34%), saturates the miniF2F benchmark, and solves 587 of 672 PutnamBench problems. It is fully open-sourced and accessible via Hugging Face and a free API, enabling practical proof engineering for users without specialized hardware.
Leanstral 1.5 undergoes a three-stage training process: mid-training, supervised fine-tuning, and reinforcement learning with CISPO. It operates in two RL environments: a multiturn environment where it refines proofs based on Lean compiler feedback, and a code agent environment where it edits files, runs commands, and uses the Lean language server to inspect goals in real time. This allows the model to handle long-horizon tasks such as completing partial proofs and building auxiliary lemmas across multiple rounds of context compaction.
The model demonstrates strong performance on formal mathematics and code verification benchmarks. On PutnamBench, it solves 587 problems at an estimated cost of $4 per problem, outperforming Seed-Prover 1.5 high by 7 problems while maintaining lower computational expenses. On FLTEval, it improves pass@1 from 21.9 to 28.9 and pass@8 from 31.9 to 43.2, surpassing Opus 4.6 at one-seventh the cost. Leanstral 1.5 also excels in test-time scaling, solving more problems as token budgets increase from 50k to 4M.
Leanstral 1.5 has identified previously unknown bugs in 57 open-source repositories, including a critical overflow issue in the zigzag decoding sign function of the datrs/varinteger library. The model’s automated pipeline, which translates Rust code to Lean and infers correctness properties, flagged 47 violated properties, with 11 indicating genuine bugs. The weights are available on Hugging Face, and the model can be accessed via a free API endpoint as leanstral-1-5, with recommended usage in Mistral Vibe.