DeepSeek-Prover-V2-671B model and plain language paper interpretation (AI version)
The DeepSeek-Prover-V2-671B model is released, ushering in a new era of AI for Science.
Core content:
1. Features and architecture of the DeepSeek-Prover-V2-671B model
2. Applications and challenges in the field of AI for Science
3. Paper interpretation: Methods for enhancing the theorem proving ability of LLMs with large-scale synthetic data
Background
- Background:
The background of this article is that the complexity of proofs in modern mathematics is increasing, making errors in peer review difficult to detect. To address these problems, formal mathematical languages such as Lean, Isabelle, and Coq have been developed to allow computers to verify proofs. However, writing formal proofs requires a lot of expertise, and the importance of automated theorem proving is increasing. - Research content:
The research content of this problem involves how to enhance the ability of LLMs in theorem proving through large-scale synthetic data. Specifically, the article proposes a method to generate a large amount of Lean 4 proof data from informal mathematical problems, and improves its theorem proving performance by fine-tuning the DeepSeekMath 7B model. - literature review:
Related work on this problem includes Polu and Sutskever (2020), Jiang et al. (2021), Han et al. (2021), Polu et al. (2022), Lample et al. (2022), Jiang et al. (2022a), Yang et al. (2024), etc. These works mainly focus on the combination of search algorithms and large language models, but lack sufficient training data. Autoformalization methods such as Wu et al. (2022) generate some synthetic data, but the scale is still insufficient.
Research Methods
This paper proposes a method to enhance the ability of LLMs in theorem proving by using large-scale synthetic data. Specifically, the research method includes the following aspects:
Automatic Formalization: Generate formal mathematical statements from high school math competition problems. We first use the DeepSeek-Prover model to translate natural language problems into formal statements for Lean 4. The initial model has difficulty with this task, so we fine-tune it using the MMA dataset, which contains natural language problem descriptions back-translated from Lean 4's mathlib3.
Quality filtering: High-quality formal statements are screened through model scoring and hypothesis rejection methods. Model scoring uses a chain thinking approach to classify statements as "excellent", "good", "above average", "average", or "poor", and excludes "average" and "poor" statements. Hypothesis rejection methods detect invalid hypotheses by trying to prove that the conclusion of the statement is false.
Statement Proof: Use the model to search for proofs of statements and improve efficiency by proving negative statements in parallel. Each proof search flow tries at most k proofs and terminates the search once a valid proof is found.
Iterative enhancement: Iterative enhancement is performed by continuously fine-tuning the model and generating new data. After each iteration, the performance of the model will be improved until the improvement becomes negligible.
Figure 1 provides an overview of the paper’s approach, showing the entire process from informal mathematical questions to formal proofs. The following is a detailed explanation of each step in the figure:
Autoformalization :
- enter
: A large collection of informal math problems (e.g., problems from high school and college math competitions). - process
: Use a deep learning-based language model (such as the DeepSeekMath-Base 7B model) to convert these informal questions into formal mathematical statements (Lean 4 language). - Output
: Preliminary formal mathematical statements. Quality Filtering :
- Model Scoring
: Use a scoring model to evaluate each formal statement and judge its quality (such as "excellent", "good", "average", "poor", "bad"). - Assumption Rejection
: Check the consistency of the assumptions of a statement by trying to prove a statement with a "False" conclusion. If the assumptions are inconsistent, the statement is rejected. - enter
: Preliminary formal mathematical statements. - process
: - Output
: High-quality formal mathematical statements. Statement Proving :
- Proof Search
: Use the DeepSeek-Prover model to try to find a proof for each formal statement. To improve efficiency, prove the original statement and its negation at the same time (i.e., prove T ⊢ P and T ⊢ ¬P in parallel). - verify
: Use the Lean 4 verifier to verify that the found proof is correct. - enter
: High-quality formal mathematical statements. - process
: - Output
: Verified formal mathematical statements and their proofs. Iterative Enhancement :
- Fine-tuning the model
: Fine-tune the DeepSeek-Prover model using newly generated proof data. - Repeat the process
: Use the fine-tuned model for the next round of automatic formalization, quality filtering, and statement proof. - enter
: Verified formal mathematical statements and their proofs. - process
: - Output
: DeepSeek-Prover models with continuously improved performance and larger-scale, high-quality formalized mathematical datasets. - Multi-step quality assurance
：Filter low-quality statements through quality scoring models and hypothesis rejection strategies, and use an iterative framework to continuously improve the quality of proofs. - Scale assurance strategy
: Speed up the proof process by proving the original statement and its negation in parallel, avoiding wasting time on unprovable statements. Automatic formalization is effective at scale, and models trained with automatically generated data perform better than models trained with mathlib data alone. The model is effective in filtering low-quality sentences, and the performance of the model trained with high-quality proof data is 4.5% higher than that of the model trained with low-quality data. The iterative enhancement strategy is effective. As the number of iterations increases, the model performance improves and the theorem proving capability is enhanced. The synthesis theorem proves that the data size is related to the model effect. The larger the data volume, the better the model performance on the miniF2F benchmark. If the original proposition can be proved, then it is correct. If the counter-proposition can be proved, then the original proposition is wrong.
Through these steps, the method in this paper can effectively generate large-scale high-quality formal mathematical data and significantly improve the performance of automatic theorem proving.
Here are 10 questions and answers about this paper, explaining its value in an easy-to-understand way:
1. What problem does this paper mainly solve?
In the field of mathematical theorem proving, large language models (LLMs) have great potential, but the lack of training data has limited progress in formal theorem proving. This paper proposes a method to address this data shortage by generating a large amount of Lean 4 proof data from high school and undergraduate mathematics competition problems.
2. How does it solve the problem of lack of data?
The paper first collects natural language problems from high school and undergraduate mathematics competitions from the Internet, and then uses a large language model to convert these natural language problems into formal statements of Lean 4. Then it filters out low-quality and invalid statements, and then uses the model to generate proofs. Finally, it uses these data to fine-tune the model and continuously iterate to improve model performance.
3. How effective is this approach?
In experiments on the Lean 4 miniF2F test set, the fine-tuned DeepSeek-Prover model achieved 46.3% complete proof generation accuracy and 52% cumulatively when using 64 samples, surpassing the baseline model GPT-4's 23.0% and a tree search reinforcement learning method's 41.0%. On the Lean 4 Formalized International Mathematical Olympiad (FIMO) benchmark, the model successfully proved 5 of 148 problems, while GPT-4 failed to prove any.
4. What is unique about the method proposed in the paper?
5. What contribution does this paper make to the field of mathematics?
It creates and open-sources a high-quality dataset of formal mathematical proofs, providing more resources for the mathematics and artificial intelligence communities, helping to further research and develop automatic theorem proving, potentially making mathematical proof verification more reliable, and providing educational resources for students and researchers.
6. What does this mean for the field of artificial intelligence?
It demonstrated the potential of large language models in automatic theorem proving, improved model performance by utilizing large-scale synthetic data, provided new ideas and methods for other related research, and promoted the development of artificial intelligence in the field of mathematical reasoning.
7. How were the experiments in the paper set up?
DeepSeek-Prover is built on the DeepSeekMath-Base 7B model, using 120 billion math-related tags for pre-training. During fine-tuning, the global batch size is set to 512, the learning rate is set to 1×10⁻⁴, and 6000 warm-up steps are included. During evaluation, it is compared with baseline models such as GPT-3.5, GPT-4, and GPT-f, and the pass@k metric is used to measure performance.
8. In addition to proving accuracy, are there any other findings from the experimental results?
Through ablation experiments, we found that:
9. What is the role of a case study in a paper?
The effectiveness of the method in practical applications was demonstrated, including successful proof of theorems and identification of inconsistent assumptions. For example, when automatically formalizing theorems, it can accurately convert mathematical problems described in natural language into formal statements of Lean 4 and prove them; it can also find incorrect assumptions in the original automatic formalization and provide counterexamples and revised versions.
10. What are the future prospects of the paper?
Currently, we focus on algebra and number theory problems in middle schools and undergraduate universities. In the future, we plan to expand the diversity of mathematical problems, improve the versatility of methods in automatic theorem proving, and further promote the development of the field of automatic theorem proving.
The unique value of this article (analogy version)
1. Train the model like “teaching AI to do math problems”
Imagine you want to teach an AI to solve a high school math competition, but you find that it can't even understand the questions, let alone solve them. The previous method was like giving the AI a bunch of "standard answers" and letting it memorize them, but it would be confused when it encountered new questions.
The approach of this paper is: first let AI learn to translate math problems into "machine-understandable language" (Lean 4), then try to solve the problem by itself, and finally correct it with the correct answer . It's like you first teach AI to understand the problem, then let it try to solve it by itself, and tell it where it went wrong if it made a mistake, and train it repeatedly until it can solve the problem independently.
2. Improve AI’s math ability by “brushing through question banks”
Previous AI training data was like "a few thin exercise books" with too few questions, so AI did not learn well enough. This paper generated 8 million math questions and answers , which is equivalent to preparing a "super question bank" for AI, allowing it to practice questions like crazy, and its problem-solving ability will naturally improve.
3. Optimize training like a "wrong question book"
In the past, when training AI, if it made a mistake, it might just discard the wrong case. But the method in this article is to sort out the wrong questions, analyze why they were wrong, and then let the AI learn again . For example, if the AI takes "all complex numbers satisfy a certain error condition" as a true proposition, it will find the contradiction by itself and eliminate the wrong case to avoid making the same mistake again next time.
4. Improve problem-solving efficiency like a "double insurance"
In the past, AI could only prove math problems one by one, which was very inefficient. The method of this paper is to prove the original proposition and its inverse proposition at the same time , just like "walking on two legs":
This will help quickly eliminate erroneous propositions, save computing resources, and improve efficiency.
Why did the author write this paper? (An analogy)
1. Solve the bottleneck problem of AI mathematical problem solving
Today’s AI is like a “primary school student” in mathematical reasoning. It can solve simple problems, but is at a loss when faced with complex math competition problems. The author found that insufficient data is the main bottleneck - there are not enough high-quality math problems for AI to learn.
So, they decided to create their own data and automatically generate 8 million math problems to give the AI enough "practice questions" to learn, just like hiring a "math private tutor" for the AI and forcing it to practice questions every day.
2. Let AI think like a mathematician
Previous AI problem solving methods were like "brute force cracking", trying all possibilities, but the efficiency was extremely low. The author hopes that AI can be like a mathematician, first understand the problem, and then deduce it logically .
Their method allows AI to first learn to translate math problems into "machine language" and then try to prove them on its own, just like teaching a person to first learn to read a problem and then think about the steps to solve it on their own, rather than just copying the answer directly.
3. Promote the development of AI automatic theorem proving
Automatic theorem proving (ATP) is like "autonomous driving" in mathematics. If AI can prove mathematical theorems by itself, it will be able to help mathematicians verify conjectures and discover new theorems in the future.
But in the past, ATP had too little data for AI to learn. This paper uses large-scale synthetic data to significantly improve AI's "mathematical driving technology", which may allow AI to play a greater role in mathematical research in the future.
4. Make AI mathematical tools accessible to more people
The author has made the data and model open source, just like making the "mathematics learning secrets" public, so that researchers around the world can use them. In this way, more people can improve AI mathematical capabilities based on this method, and even apply it to education, scientific research and other fields.
Summary (one sentence version)
The value of this paper lies in: using AI to automatically create 8 million math problems, letting AI practice them like crazy, greatly improving its problem-solving ability, and open-sourcing data and methods to promote the cross-border development of mathematics + AI . The authors did this because insufficient data is the bottleneck of AI mathematical reasoning, and they found a way to efficiently create data , bringing AI one step closer to becoming a "mathematical expert."