DeepSeek Announces Full Open-Source Release of Internal Inference Framework DeepInfer

DeepSeek announced that it will fully open-source its internal inference framework DeepInfer, while simultaneously releasing the production-grade token encoder-decoder component deepseek-recipe, to help developers and enterprises optimize the throughput, latency, and deployment costs of large-scale model inference. DeepInfer integrates high-performance operators and a deeply optimized inference engine, supporting capabilities such as distributed inference, speculative decoding, SWA Bounded Replay, and structured output, while maintaining compatibility with NVIDIA and Huawei Ascend platforms. DeepSeek noted that it has already realized high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash based on DeepInfer, and future inference code for new models along with subsequent optimizations will continue to be updated within this framework.