Quantization Methods Compared: GGUF, AWQ, GPTQ, EXL2, NVFP4

What is Quantization?

Neural network weights are typically stored as 16-bit floating point numbers (FP16 or BF16). An 8-billion parameter model needs 16 GB of memory just for the weights.

Quantization reduces the precision of these numbers — from 16-bit down to 8, 6, 4, or even 2 bits — making models smaller and faster at the cost of some quality.

The speedup comes from the memory wall — LLM inference is bottlenecked by how fast you can stream weights from VRAM, not by computation. Smaller weights = faster streaming.

Why Inference is Memory-Bound

A modern GPU like the RTX 5090 has massive compute power:

CUDA cores want:    103 TB/s of weight data
VRAM delivers:        1.8 TB/s
Gap:                  57x — cores idle 98% of the time

During token generation, the entire model is read from memory for each token. Quantization reduces the amount of data to read, directly improving throughput:

These are theoretical maximums. Real-world performance is lower due to overhead, but the relative ratios hold.

The Formats

GGUF (llama.cpp)

The most widely used format. GGUF uses mixed-precision integer quantization with multiple sub-formats:

Pros: Universal compatibility (Ollama, llama.cpp, kobold.cpp), well-tested, many options Cons: Uses integer tensor cores, leaving FP4/FP8 cores unused on newer GPUs

GPTQ

GPT-Quantized. A calibration-based 4-bit integer format that uses sample data to minimize quantization error.

Pros: Good quality at 4-bit, works with vLLM for multi-LoRA Cons: Requires calibration step, slower to create

GPTQ-Int4 is the best path for vLLM + LoRA — NVFP4 doesn't support LoRA adapters yet.

AWQ

Activation-Aware Weight Quantization. Similar to GPTQ but protects important weights identified by analyzing activations.

Pros: Slightly better quality than GPTQ at 4-bit Cons: Smaller ecosystem, fewer pre-quantized models available

EXL2 (ExLlama v2)

A flexible format that allows per-layer bit allocation — giving more bits to important layers and fewer to redundant ones.

Pros: Best quality per bit, fine-grained control Cons: Limited to ExLlama v2 runtime, no vLLM support

NVFP4 (Blackwell FP4)

NVIDIA's 4-bit floating point format, native to Blackwell GPUs (RTX 5090, B100, B200).

Pros: Uses dedicated FP4 tensor cores (not integer), hardware-native performance Cons: Hard reasoning degradation, Blackwell only, no LoRA support yet

Benchmark data from Qwen3-8B:

The drop on hard reasoning tasks is significant. For business applications that require accurate numerical reasoning (prices, calculations), this matters.

Head-to-Head: Speed vs Quality

All benchmarks on RTX 5090, Qwen3-8B, single user:

*vLLM speeds include framework overhead; aggregate throughput is higher with concurrent users.

Quantization and Non-English Languages

A critical consideration for multilingual deployments: aggressive quantization hurts low-resource languages more.

LLMs allocate weight capacity proportional to training data volume. English dominates training corpora (40-60%), so English tokens get the most model capacity. For languages with less training data:

• 6-bit (Q6_K): Preserves 98-99% quality across languages
• 4-bit (Q4_K_M/GPTQ): Drops to 90-95% for non-English
• 4-bit (NVFP4): Can drop to 80-92% on hard reasoning in non-English

If you're serving customers in languages other than English, Q6_K is the safest choice. The extra 2 GB of VRAM is a small price for maintaining quality across all languages.

Decision Matrix

Practical Advice

1. Start with Q6_K GGUF. It's the best all-around choice: excellent quality, fast inference, works everywhere.

2. Drop to Q4_K_M only if VRAM forces you to. The 3% quality loss is noticeable in edge cases.

3. Use GPTQ-Int4 for multi-LoRA vLLM deployments. It's the only 4-bit format with LoRA adapter support.

4. NVFP4 is for multi-user production only. Don't use it for single-user chat — Ollama with Q6_K is 2.4x faster.

5. Test with your actual data. Benchmark numbers vary by model family, language, and task type. Always validate on your specific use case before deploying.