quantization

Quantization is a compression technique that lowers the numerical precision of a model’s weights, storing them in smaller data types such as 8-bit or 4-bit integers instead of 16- or 32-bit floating point. The result is a much smaller large language model that needs less memory and runs inference faster, at the cost of some accuracy.

Mapping a wide floating-point range onto a handful of integer levels introduces rounding error, so the goal is to shed precision where the model can absorb it. Weight-only schemes shrink only the stored parameters, while others also quantize the activations, the intermediate values that flow between the model’s layers, to speed up the arithmetic itself.

This precision tradeoff is easiest to see by adjusting the bit width directly. As the bits drop, a smooth high-precision curve snaps onto fewer discrete levels, and the shaded gap between the curves is the rounding error traded for a smaller, faster model.

Interactive diagram — enable JavaScript to view.

Two broad approaches trade off effort against quality. Post-training quantization (PTQ) converts an already-trained model in a single pass, while quantization-aware training (QAT) simulates the rounding during training so the weights adapt, usually preserving more accuracy at very low bit widths.

Common formats include INT8, INT4, and FP8, and popular methods such as GPTQ, AWQ, and bitsandbytes make it practical to run large models on a single consumer GPU.

How to Run Large Language Models Locally With Ollama

Tutorial

How to Use Ollama to Run Large Language Models Locally

Learn how to use Ollama to run large language models locally. Install it, pull models, and start chatting from your terminal without needing API keys.

intermediate ai tools

For additional information on related topics, take a look at the following resources:

By Martin Breuss • Updated Aug. 19, 2026
AI Coding Glossary Share Feedback