Open sourceNov 4, 2026, 8:08 AM EST

llama.cpp 0.9.0 adds FP8 E4M3 quantization and four model architectures

The llama.cpp project released version 0.9.0 of its open-source local inference engine. The update adds an OCP FP8 E4M3 tensor type in saturating mode, with conversion helpers and a new ftype, plus a flash_attn_ext_rows op that makes the unified KV cache cheaper for multi-sequence inference. A batched tensor copy API replaces the older _2d calls, implemented for Metal and CUDA. Four architectures are now supported: Clover-35B-A3B-Preview in autoregressive mode, Limite 1B Violetto, Metis-1.6, and Mistral Large 4. Gemma 4 E2B/E4B get prefill optimizations, and ggml v0.29.0 brings NCCL multi-node tensor parallelism.

Sources

Written by AIToolMaps from the source above and checked against it sentence by sentence.

Comments

No comments yet