Up to 3.2x Faster Inference with LFM2.5-DSpark

Hugging Face has introduced DSpark draft model checkpoints for three models in the LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These draft models use speculative decoding, a technique that introduces a lightweight draft model to generate candidate tokens, which the target model verifies in a single forward pass. This method reduces memory-bound latency by sharing the cost of loading weights across multiple tokens, improving inference speed with minimal memory overhead.

The draft models are trained using a diverse dataset covering supervised fine-tuning, chat, code, and function-calling data. Each draft model consists of approximately 300 million parameters and is optimized for high acceptance rates rather than low loss. The emitted sequences remain identical to baseline greedy decoding, ensuring no change in benchmark accuracy. The draft models support day-one integration with llama.cpp and SGLang, with implementations built on their official codebases.

Performance benchmarks conducted on an M4 Max MacBook Pro with FP16 GGUF weights and a single H100 80 GB GPU in BF16 show significant throughput improvements. For LFM2.5-2.6B, the speedup is particularly notable on the MacBook, achieving around 140 tokens per second and reducing latency by 57% on average across multi-tool scenarios. LFM2.5-1.2B-Instruct exhibits variable speedup up to 52%, depending on the text distribution, while LFM2.5-8B-A1B achieves an 18% average improvement due to current MoE implementation constraints.

To use the DSpark draft models with SGLang, a build with DSpark support for LFM2.5 targets is required, and the target model must be launched with the draft attached. For llama.cpp, a respective build is needed, with speculative decoding parameters read from metadata. The output remains exact, as the target model verifies every proposed token, ensuring greedy output matches the baseline.