MPS in PyTorch 2.15

PyTorch 2.15 adds FP8 tensor support on MPS, enabling memory savings with torch.float8_e4m3fn tensors that use one byte per value, half the storage of FP16 or BF16. The release also introduces F.scaled_mm for tensorwise and rowwise scaling, expanding low-precision computation options for Mac users. FP8 operations are now available alongside existing precision formats, providing an additional tool for optimizing memory usage during model inference and training.

Language-model decoding performance improves significantly in PyTorch 2.15, particularly for dense linear layers and mixture-of-experts layers. The update connects F.linear to MPS GEMV kernels, delivering a 1.23× geometric-mean speedup across 96 BF16 layer shapes from real models. Mixture-of-experts layers benefit from native grouped matmul implementations, reducing CPU-GPU data transfers and improving token throughput by about 1.3× on an M3 Ultra with torch.compile.

Complex-valued computations receive native MPS support in PyTorch 2.15, including complex64 LU factorization, LU solve, and triangular solve. Small matrix operations, such as FP32 matrix inversion for camera geometry, now run on dedicated Metal kernels for batches up to 4×4. Larger problems like QR factorization and SVD also see improvements, with better handling of numerical extremes and more reliable pseudoinverse and least-squares results.

New Metal kernels target audio, video, and training workloads, including faster softmax and log_softmax operations that reduce latency for large vocabularies. A constant-padding optimization for video decoding in Wan-AI/Wan2.2-TI2V-5B-Diffusers cuts peak driver memory from 270 GiB to 8 GiB during a 28-tile decode. Additional operations now run on MPS, and attention correctness fixes address batch and head broadcasting in scaled dot-product attention.