Can You Run Llama 3.3 70B on 4× NVIDIA RTX 4090?

The Llama 3.3 70B (70.6B Dense) contains approximately 70.6 billion parameters. At FP16 (2 bytes per parameter), the raw weight matrix occupies 140 GB. With a 15% CUDA kernel overhead factor, effective VRAM for weights alone is 140 GB. The 4× NVIDIA RTX 4090 provides 24 GB GDDR6X each (96 GB total) of memory, leaving insufficient headroom — quantization to INT8 or INT4 is mandatory for KV-cache and activation tensors.

At INT8 quantization (1 byte per parameter), the weight footprint drops to 70 GB, requiring a tensor parallel degree of approximately 3× across 4× NVIDIA RTX 4090 instances. The estimated inference throughput is ~45 tok/s (INT4, TP=4, batch=1), which translates to a cost-per-million-output-tokens of roughly $2.00 at current spot rates. For production deployments, vLLM (tensor-parallel-size=4, INT4 only) with PagedAttention is the recommended serving stack.