AWS Neuron 2.32 introduces expanded NKI programming, MXFP8 training kernels, and variable-size collectives for Trn2 and Trn
AWS released Neuron 2.32.0 with expanded NKI programming, MXFP8 training kernels, and variable-size collectives for Trn2 and Trn3 instances, enhancing model training efficiency.
AWS Neuron 2.32.0 introduces NKI 0.6.0, featuring an on-device top-K instruction and variable-length all-gather operations for kernels with differently sized tensors. The update adds 13 new NKI Library kernels for Mixture of Experts (MoE) training and sparse attention, including support for DeepSeek-V3.2’s sparse multi-head latent attention. The release also includes a new Neuron Agentic Development skill that ports transformer models to the vLLM Neuron plugin, expanding compatibility with existing frameworks.
The update introduces MXFP8 attention for decode steps and a matched MXFP8 forward and backward pass, enabling blockwise MoE layers to train end-to-end in MXFP8. PyTorch reference implementations are provided for 22 additional kernels, supporting broader model architectures. The Neuron Runtime now supports variable-size all-gather, reduce-scatter, and all-to-all collectives on Trn2 and Trn3 instances, allowing flexible data distribution across ranks.
The vLLM Neuron plugin has been updated to vLLM 0.24.0 and is included in all Neuron Deep Learning AMIs and Deep Learning Containers. The Neuron Compiler gains explicit control over 64-bit integer compilation, while Neuron Explorer adds per-core host CPU utilization tracking to the System Trace Viewer for improved debugging.
AWS Neuron 2.32.0 is available in all AWS Regions where Amazon EC2 Trn1, Trn2, Trn3, Inf2, and Inf1 instances are offered. Users can refer to the AWS Region table for detailed availability information across supported regions.