Self-Host DeepSeek V4 Flash
13B active parameters means you can actually run this one. Here is the hardware, the software stack, and the math on when self-hosting beats the API.
Why Self-Hosting V4 Flash Is Practical
Most frontier models are too large to self-host economically. V4 Flash is different: 13B active parameters per token (out of 284B total in the MoE architecture) means only a fraction of the model is loaded for any given request. Two A100 80GB GPUs handle it comfortably with tensor parallelism.
The economics work at scale: community benchmarks show 400 tokens per second at ~$10/hour on rental hardware. That is roughly $6.94 per million tokens all-in — more than the API for light use, but at sustained throughput you stop paying per token and start paying per hour.
Three Serving Frameworks
vLLM (recommended)
vllm serve deepseek-ai/DeepSeek-V4-Flash --tensor-parallel-size 2
Hardware: 2× A100 80GB or equivalent Speed: ~400 tps reported
SGLang
python -m sglang.launch_server --model deepseek-ai/DeepSeek-V4-Flash --tp 2
Hardware: 2× A100 80GB or equivalent Speed: Comparable to vLLM
llama.cpp (GGUF)
Download GGUF from community quants, run with llama-server
Hardware: 64GB+ RAM for Q4 quants Speed: Slower but runs on consumer hardware
API vs Self-Hosting: When to Switch
# Break-even calculation
API cost: $0.14/M input + $0.28/M output ≈ $0.21/M blended
Self-host: $10/h ÷ 1.44M tokens/h = $6.94/M all-in
# API is cheaper below ~70M tokens/hour
# Self-host wins above that — and you control the data
Use the API when…
- Volume is under 70M tokens/hour
- You want zero infrastructure management
- You need burst capacity without provisioning
Self-host when…
- Sustained throughput exceeds 70M tokens/hour
- Data must stay on your infrastructure
- You need custom quantization or fine-tuning
Self-Hosting FAQ
What is the minimum hardware for self-hosting?
Two A100 80GB GPUs (or equivalent VRAM) for the full BF16 model with tensor parallelism. For GGUF quantized versions, 64GB+ system RAM can work with llama.cpp on consumer hardware at lower speed.
Is self-hosting cheaper than the API?
Only at sustained high throughput. The crossover is roughly 70M tokens per hour. Below that, the API at $0.14/M is simpler and cheaper. Above that, a $10/hour GPU rental producing 1.44M tokens/hour becomes more economical.
Which serving framework is best?
vLLM is the most battle-tested for MoE models and offers the best throughput. SGLang is a strong alternative with similar performance. llama.cpp with GGUF quants is best for consumer hardware or Mac inference.
Can I run V4 Flash on a Mac?
With GGUF quantization, yes — a Mac with 64GB+ unified memory can run Q4 quants. Expect slower inference than GPU servers but fully functional for development and testing.