Tree-Ising repairs language models with bit-level Ising weight edits
A paper posted to arXiv describes Tree-Ising, a method that treats repairing a language model's reasoning loops as a sparse discrete optimization problem over the model's own BF16 weights. Instead of having an LLM agent propose weight edits, the authors use forward statistics to build an objective that weakens a looping direction and strengthens a concluding one while keeping the layer's output on useful tokens. The objective splits into independent QUBO problems, one per output row, solved on a GPU with simulated annealing or simulated bifurcation. On the two layers an earlier agent edit targeted, Tree-Ising reached 54.6% on MATH-500 against 51.7% for that edit, with the base model at 7.9%. The paper is available now; no code release is stated.
Why it matters
The repair needs four forward-pass requests and zero LLM tokens, with the annealing run costing about $0.90 in GPU time, and it changed 13% to 17% fewer weights than the agent edit. If the results hold, fixing specific model behaviors could become a small optimization job rather than a long agent session.
Sources
- PrimaryHugging Face Papers
Written by AIToolMaps from the source above and checked against it sentence by sentence.