1. Introduction

With the continued popularity of DeepSeek-V3 and R1, various domestic manufacturers have set off a wave of access or adaptation to DeepSeek models, which is undoubtedly a good thing. At the same time, more and more users or companies want to deploy DeepSeek models locally to experience the convenient functions or secondary development it brings, but the DeepSeek model has certain requirements for hardware when deployed. The following summarizes the hardware requirements for the deployment of the basic model of DeepSeek-R1 and the distillation model.

2. Local deployment configuration requirements for DeepSeek-R1 basic model

General recommendations for local deployment and use of DeepSeek-R1 models

• •  Quantization optimization : Using 4-bit/8-bit quantization can reduce video memory usage by 30-50%.
• •  Inference framework : Improve efficiency by using acceleration libraries such as vLLM and TensorRT.
• •  Cloud deployment : 70B/671B recommends giving priority to cloud services to elastically scale resources.
• •  Power consumption note : 32B+ models require a high power supply (1000W+) and cooling system.

3. Local deployment configuration requirements for the DeepSeek-R1 distillation model and its quantized version

When using the distillation model, the CPU configuration should be as close to or slightly lower than that of the DeepSeek-R1 model with the same parameter scale. I have also successfully deployed DeepSeek-R1-Distill-Llama-70B locally using an NVIDIA A40, and tried to use ollama directly or use a python program to ask it questions. The speed and accuracy of its answers (only for ordinary questions asked) are acceptable.

In addition, when deploying and using AI models, we should start with a smaller model for trial deployment according to our needs, and then upgrade to a model with a higher parameter scale if necessary, so as to meet the needs while avoiding waste of resources.