To install this model locally in the shortest time, opt for a direct curl execution.
Carefully read and apply the steps described below.
The download manager will automatically pull several gigabytes of data.
The installer will automatically analyze your hardware and select the optimal configuration.
DeepSeek-R1-0528-NVFP4-v2 is a cutting-edge large language model designed to revolutionize low-precision inference on NVIDIA’s Hopper architecture. Leveraging the NVFP4 data type, this model achieves remarkable throughput while maintaining state-of-the-art accuracy. With a parameter count of 180B and training on over 5 trillion tokens, DeepSeek-R1-0528-NVFP4-v2 enables robust reasoning across diverse domains. Its inference latency averages 23ms per token on a single A100-80GB, making it suitable for real-time applications. This design incorporates mixture-of-experts layers that dynamically route queries to specialized subnetworks, improving both efficiency and scalability.
•
•
| Technical Specifications | Values |
|---|---|
| Parameter Count | 180B |
| Training Tokens | 5 trillion |
| Inference Latency | 23ms/token |
| Precision | NVFP4 |
• Q: What is the NVFP4 data type, and how does it impact performance?A: The NVFP4 data type enables high-performance inference on NVIDIA’s Hopper architecture. This results in improved throughput while maintaining state-of-the-art accuracy.• Q: How does DeepSeek-R1-0528-NVFP4-v2 improve reasoning across diverse domains?A: By leveraging mixture-of-experts layers, this model dynamically routes queries to specialized subnetworks, improving efficiency and scalability.• Q: What are the implications of 23ms per token inference latency for real-time applications?A: Despite its high performance, DeepSeek-R1-0528-NVFP4-v2’s inference latency makes it suitable for real-time applications that require rapid processing.