LLM VRAM Calculator: GPU Requirements for Llama, Qwen, DeepSeek & Claude in 2026

Direct answer

GPU sizing starts with weights — parameters × bytes per parameter — then adds KV cache, runtime overhead, and headroom. FP8 stores every parameter in 1 byte, so a 70B model is 71 GB of weights (not 140; that is the FP16 figure). At 32K context a 70B FP8 serving instance needs ~81 GB total, which is why it lands on 1× H200 or 2× H100.

Model size	FP8 weights	Typical in-service total (32K ctx)	Single GPU?	Minimum GPU
7B–8B (Llama 3.1 8B, Qwen 2.5 7B)	8 GB	~13 GB	✅ RTX 4090, L40S	24 GB+
12B–14B (Gemma 3 12B, Qwen Coder 14B)	12–14 GB	~17–20 GB	✅ RTX 4090, L40S	24 GB+
22B–27B (Codestral 22B, Gemma 2 27B)	22–27 GB	~27–33 GB	✅ L40S	48 GB
32B (Qwen 2.5 Coder 32B)	33 GB	~41 GB	✅ L40S, H100	48 GB
70B (Llama 3.3 70B, Qwen 2.5 72B)	71–73 GB	~81–87 GB	❌ (80 GB is marginal)	H200 (141 GB) or 2× H100
104B (Command R+)	104 GB	~117 GB	❌	2× H100 or 1× H200 (141 GB, tight)
141B (Mixtral 8×22B)	141 GB	~158 GB	❌	2× H100 (tight) or 2× H200
340B (Nemotron-4 340B)	340 GB	~380 GB	❌	5× H100 or 3× H200
405B (Llama 3.1 405B)	405 GB	~482 GB @32K	❌	4× H200 (32K) / 5× H200 (128K)
671B (DeepSeek R1)	671 GB	786 GB @128K	❌	6× H200 (min) or 5× B200

Sources: weights from the deterministic formula (parameters × bytes/param) and the model architecture registry (minVramFp8Gb = 71 GB for Llama 70B, 671 GB for DeepSeek R1); totals from the canonical VRAM engine (weights + KV cache + 1.2 GB CUDA overhead + 0.4 GB activations + 10% fragmentation headroom), batch=1, 32K context unless noted. GPU VRAM from the GPU spec database.

How to calculate VRAM requirements

VRAM = weights + KV cache + activations + overhead

Formula:

weights      = params × bytes_per_param        (FP8 = 1, FP16 = 2, INT4 = 0.5)
kv_cache     = 2 × layers × kv_heads × head_dim × context × bytes_per_kv_element
total        = (weights + kv_cache + 1.2 GB CUDA + 0.4 GB activations) × 1.10

The 10% headroom covers allocator fragmentation — serving frameworks routinely reserve it, and skipping it is how "fits on paper" deployments OOM.

Quantization formats

Format	Bytes/param	Weights vs FP16	Quality impact	Recommended for
FP16/BF16	2	baseline	none	training, accuracy-critical eval
FP8	1	−50%	negligible in practice for serving	production serving
INT8	1	−50%	negligible	supported on older hardware
INT4 (AWQ/GGUF)	0.5	−75%	measurable — benchmark on your task	memory-constrained deployment

Note: perplexity deltas from quantization are model- and dataset-dependent; treat any single published percentage as an estimate, not a guarantee. See the DeepSeek R1 FP8 vs INT4 analysis for the measurement approach.

KV cache calculation

KV cache (GB) = 2 × layers × kv_heads × head_dim × context_length × bytes_per_element ÷ 1,073,741,824

Worked examples at 128K context, FP8 KV (1 byte/element), batch=1, using the architecture presets in our VRAM engine:

Model	Layers (engine preset)	KV @128K
Llama 3.1 8B	32	8 GB
Qwen 2.5 Coder 32B	52	13 GB
Llama 3.3 70B	28 (see note)	7 GB — treat as 7–20 GB
Qwen 2.5 72B	80	20 GB
Llama 3.1 405B	126	126 GB
DeepSeek R1 (MLA)	compressed KV	42 GB

Note on Llama 70B layers: our registry lists 28 layers for Llama 3.3 70B and 80 for Llama 3.1 70B — the same architecture. Until that record is reconciled, size 70B KV cache against the 80-layer figure (20 GB @128K) rather than the low one. DeepSeek R1 uses MLA compressed KV (~0.5 KB/token/layer), which is why its KV cache is only 42 GB despite 671B parameters.

VRAM chart by model

Model	Params	FP8 weights	INT4 weights	Total FP8 @32K	Needs GPU
Llama 3.1 8B	8.03B	8 GB	4 GB	~13 GB	RTX 4090 (24 GB) ✅
Qwen 2.5 7B	7.6B	8 GB	4 GB	~13 GB	RTX 4090 (24 GB) ✅
Gemma 3 12B	12B	12 GB	6 GB	~17 GB	RTX 4090, L40S ✅
Codestral 22B	22B	22 GB	11 GB	~27 GB	L40S (48 GB) ✅
Qwen 2.5 Coder 32B	32.5B	33 GB	16 GB	~41 GB	L40S, H100 ✅
Llama 3.3 70B	70.6B	71 GB	35 GB	~81 GB	H200 (141 GB) ✅; 2× H100 ✅
Qwen 2.5 72B	72.7B	73 GB	36 GB	~87 GB	H200 ✅
Mistral Small 24B	24B	24 GB	12 GB	~30 GB	L40S ✅
Mistral Large 123B	123B	123 GB	62 GB	~138 GB	2× H100 ✅; 1× H200 (tight) ✅
Command R+ 104B	104B	104 GB	52 GB	~117 GB	2× H100 ✅; 1× H200 ✅
Mixtral 8×22B	141B	141 GB	70 GB	~158 GB	2× H100 ✅ (tight)
Llama 3.1 405B	405B	405 GB	203 GB	~482 GB @32K	4× H200 ✅ (32K); 5× H200 @128K
DeepSeek R1 671B	671B	671 GB	336 GB	786 GB @128K	6× H200 ✅ (min); 8× H200 comfortable

Key insight: INT4 puts 70B weights in 35 GB (~43 GB in service at 32K) — comfortable on a single H100. FP8 at 71 GB weights / ~81 GB in service is what pushes 70B onto H200.

GPU VRAM reference

GPU	VRAM	FP8 TFLOPS	Cost (Vast.ai, 2026-10-03)	Recommended use case
RTX 4090	24 GB	165	$0.34/hr	7B–8B FP8/INT4, prototyping
L40S	48 GB	733	$0.69/hr	up to ~32B FP8, batch inference
H100 SXM5	80 GB	1979	$1.89/hr	70B INT4 single-GPU; 32B FP8
H200 SXM5	141 GB	1979	$2.79/hr	70B–120B FP8 single-GPU
B200 SXM	192 GB	2250	$3.99/hr	123B–190B FP8; 671B INT4 (5×)

Sources: GPU specs from the GPU spec database; pricing from providers.json (observed 2026-10-03).

Model-by-model VRAM guide

Llama 3.3 70B (70.6B parameters)

• FP8: 71 GB weights; ~81 GB total at 32K context
• FP16: 141 GB weights; ~161 GB total at 32K
• INT4: 35 GB weights; ~43 GB total at 32K
• Recommended GPU: 1× H200 (141 GB) ✅ or 2× H100 ✅ — FP8 does not fit a single 80 GB H100 once KV cache and overhead are included
• Free API: Groq (Llama 3.3 70B, 30 RPM, no credit card)
• Calculator: Llama 3.3 70B VRAM

DeepSeek R1 671B (671B parameters, 37B active MoE)

• FP8: 671 GB weights; 786 GB total at 128K context (MLA KV cache = 42 GB)
• INT4: 336 GB weights; 417 GB total at 128K
• FP16: 1,342 GB weights; 1,524 GB total at 128K
• Recommended GPU: 8× H200 (1,128 GB) ✅ for FP8; 5× B200 (960 GB) ✅ for FP8; 3× H200 (423 GB) ✅ or 4× B200 (768 GB) ✅ for INT4
• API pricing: DeepSeek API ($0.55/M in, $2.19/M out)
• Calculator: DeepSeek R1 VRAM

Llama 3.1 8B (8.03B parameters)

• FP8: 8 GB weights; ~13 GB total at 32K, ~19 GB at 128K
• INT4: 4 GB weights; ~8 GB total at 32K
• Recommended GPU: RTX 4090 (24 GB) ✅ — fits on a single consumer GPU
• Free API: Groq (30 RPM, 14,400 requests/day)
• Calculator: Llama 3.1 8B VRAM

Qwen 2.5 Coder 32B (32.5B parameters)

• FP8: 33 GB weights; ~41 GB total at 32K, ~52 GB at 128K
• INT4: 16 GB weights; ~23 GB total at 32K
• Recommended GPU: 1× L40S (48 GB) ✅ or 1× H100 ✅
• Free API: Groq (Qwen 2.5 Coder 32B, 30 RPM — listed in our free API dataset)
• Calculator: Qwen 2.5 Coder 32B VRAM

Claude 3.5 Sonnet (Anthropic, API only)

• Weights: closed — no VRAM sizing is possible; there are no verifiable parameter counts to publish
• API pricing: $3/M input, $15/M output
• Free tier: not available — requires credit card

128K context VRAM

For long-context workloads the KV cache stops being negligible. FP8 totals at 128K context, batch=1:

Model	FP8 weights	KV cache (128K)	Total FP8 @128K
7B–8B	8 GB	8 GB	~19 GB
22B–32B	22–33 GB	~7–13 GB	~30–52 GB
70B	71 GB	7–20 GB (layer record note above)	~87–101 GB
405B	405 GB	126 GB	~586 GB
671B (MoE, MLA)	671 GB	42 GB	786 GB

Methodology: canonical VRAM engine, FP8 KV elements (1 byte), architecture presets as documented per row. The old version of this table listed FP16 weight figures under an FP8 header and understated KV cache by an order of magnitude — both corrected.

What changes the result?

1. Quantization: INT4 cuts 70B weights to 35 GB — a single H100 goes from ❌ to ✅
2. PagedAttention / KV offloading: lets you page KV cache to CPU memory — functional on 80 GB GPUs, at a throughput cost
3. Tensor parallelism: 2× H100 = 160 GB aggregate for 70B; 8× H200 = 1,128 GB for 671B
4. Batching: larger batches multiply KV cache (per-sequence) and activation memory — size headroom per target batch, not just per single stream
5. MoE models: 671B MoE activates only 37B parameters per token but must resident-load all experts — 671 GB at FP8 regardless of active count

Alternatives

• For 70B models: H200 ($2.79/hr) vs B200 ($3.99/hr) — compare GPUs
• For 32B models: L40S ($0.69/hr) runs 32B FP8 in ~41 GB — check L40S pricing
• For prototyping: RTX 4090 ($0.34/hr) for 7B–14B — check RTX 4090 pricing
• For production 671B+: B200 cluster — B200 pricing

Conclusion

In-service VRAM (FP8, 32K)	Suitable models	Minimum GPU	Cost/hr (Vast.ai)
≤ 24 GB	7B–14B	RTX 4090	$0.34
≤ 48 GB	up to ~32B	L40S	$0.69
≤ 80 GB	70B in INT4; ≤32B FP8	H100	$1.89
≤ 141 GB	70B–120B FP8	H200	$2.79
≤ 192 GB	up to ~190B FP8	B200	$3.99
> 192 GB	405B–671B+	multi-GPU (8× H200 = $22.32)	—

Calculate exact VRAM for your model and context window with the GPU VRAM Calculator, or browse model pages for per-model GPU compatibility data.

Related resources

• Inference Cost Calculator — tokens/sec per $/hr by GPU and model
• Llama 3.3 70B Model Page — VRAM, API pricing, GPU match
• DeepSeek R1 Model Page — 671B MoE VRAM requirements
• H200 Cloud Pricing — live rates for the 141 GB GPU
• B200 Blackwell Pricing — rates for the 192 GB GPU
• Free LLM APIs (No Credit Card) — alternative to self-hosting
• What is VRAM? — educational guide on GPU memory and LLM requirements