FP8, INT8, INT4) tells you precision.
Quantization method tells you how that precision was applied.
Why methods matter
TwoINT4 variants can behave very differently because method choices affect:
- Accuracy retention
- Runtime compatibility
- Memory layout and serving performance
- Calibration requirements
Common method families
Method vs format example
INT4 is the precision target.AWQ INT4 and GPTQ INT4 are different realizations of that target.
Practical guidance
- Pick runtime first (what you can deploy reliably).
- Filter to method families that runtime supports.
- Benchmark multiple quantization methods for the same precision target.
- Track failure modes, not just average benchmark scores.
Evaluation checklist
- Structured output correctness
- Long-context faithfulness
- Tool call reliability
- Safety/moderation behavior
- Tail latency under load