Choose the right quantization format for your workload: INT4 for consumer GPUs, FP8 for datacenter H100, and AWQ for optimized throughput.
Choose the right quantization format for your workload: INT4 for consumer GPUs, FP8 for datacenter H100, and AWQ for optimized throughput.
FP8 uses 1 byte per weight element, halving VRAM vs FP16. Native Tensor Core support on Hopper GPUs delivers 2x throughput at minimal quality loss.
INT4 quantization uses 0.5 bytes per weight. AWQ (Activation-Aware Weight Quantization) preserves quality better than naive INT4 by protecting high-magnitude weights.
VRAM measurements derived from model-registry.json minVramFp16Gb and minVramInt4Gb fields. Throughput figures from gpu-specs.ts fp8Tflops measurements.