How-To Guides
Deployment runbooks, quantization guides, and production inference setup for LLMs.
How-To8 min read
How to Run Llama 3.3 70B Locally: VRAM, Quantization & Deployment
Run Llama 3.3 70B at home with INT4 quantization on a dual RTX 4090 setup, or deploy FP8 on a single H100 SXM5 via vLLM.
rtx-4090-cloud-rentalh100-sxm5-cloud-pricingl40s-cloud-pricing
How-To
Production vLLM Deployment: PagedAttention, KV-Cache & Continuous Batching
To run Llama 3.3 70B in production with vLLM, allocate 2x 80GB GPUs (Tensor Parallelism = 2) or a single H200 (141GB) using FP8 precision. Configure --gpu-memory-utilization 0.92 and --kv-cache-dtype fp8 to maximize concurrent batch slots.
h100-sxm5-cloud-pricingh200-cloud-pricingl40s-cloud-pricing