Below is a list of significant papers by DeepSeek detailing advancements in large language models (LLMs). Each paper includes a brief description and highlights upcoming deep dives.
Release Date: November 29, 2023
This foundational paper explores scaling laws and the trade-offs between data and model size, establishing the groundwork for subsequent models.
Release Date: May 2024
This paper introduces a Mixture-of-Experts (MoE) architecture, enhancing performance while reducing training costs by 42%.
Release Date: December 2024
This report discusses the scaling of sparse MoE networks to 671 billion parameters, utilizing mixed precision training and HPC co-design strategies.
Release Date: January 20, 2025
The R1 model enhances reasoning capabilities through large-scale reinforcement learning, competing directly with leading models like OpenAI's o1.
Release Date: April 2024
This paper presents methods to improve mathematical reasoning in LLMs, introducing the Group Relative Policy Optimization (GRPO) algorithm.
Focuses on enhancing theorem proving capabilities in language models using synthetic data for training.
This paper details advancements in code-related tasks with an emphasis on open-source methodologies, improving upon earlier coding models.
Discusses the integration and benefits of the Mixture-of-Experts approach within the DeepSeek framework.