Can I run DeepSeek-R1-Distill-Qwen-7B on a NVIDIA H200?

Yes.Runs

You can run this comfortably. On a NVIDIA H200 (141 GB VRAM), DeepSeek-R1-Distill-Qwen-7B needs about 19 GB (FP16) and should generate at roughly ~150 tok/s. Recommended: Run on your GPU with FP16.

deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

Why

At full precision this model needs ~19 GB, which fits your GPU VRAM.

How to run DeepSeek-R1-Distill-Qwen-7B on a H200

Serve with vLLM (OpenAI-compatible API)

# Fast production serving on http://localhost:8000/v1
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-7B

Python (Transformers)

from transformers import pipeline
pipe = pipeline("text-generation", model="deepseek-ai/DeepSeek-R1-Distill-Qwen-7B", device_map="auto")
print(pipe("Hello", max_new_tokens=50))

Try the full analyzer

DeepSeek-R1-Distill-Qwen-7B on other GPUs

RTX 509032 GB VRAM RTX 508016 GB VRAM RTX 5070 Ti16 GB VRAM RTX 507012 GB VRAM RTX 409024 GB VRAM RTX 4080 Super16 GB VRAM

Other models on a H200

Qwen2.5-0.5Bon H200 GPT-2on H200 Phi-3-minion H200 Qwen2.5-Coderon H200 Mistral-7Bon H200 Llama-3.1-8Bon H200