GGUF downloads make local LLMs smaller by storing their weights at reduced precision. Model distributor Bartowski has tested a way to choose that precision separately for each tensor, rather than relying entirely on the usual rules for a named quantization format.
Working with Qwen3.5-0.8B and Qwen3.5-4B, Bartowski ran more than 1,000 quantization configurations over roughly 96 hours. The experiments measured how each compressed version changed the original model's next-token probability distribution on WikiText-2. Embedding tensors proved particularly sensitive, while some attention projections and the first and last layers also benefited from more precision.
A solver then assigned extra bits where the measurements suggested they helped most. In one Qwen3.5-4B example, the Q3_K_M variant became about 15% smaller. Another Q4_K_M configuration was a little worse on the measured divergence score but more than 5% smaller, offering a different quality-per-bit trade-off.
The tests measured changes in token probabilities, not performance on chats, coding tasks or creative work. Bartowski also checked trends in other model families, but the posted results cannot establish that every mapped GGUF download will outperform existing versions.