Summary of experience in building a local privatized RAG knowledge base

【Project Background】

In February 2025, the domestic AI field ushered in a phenomenal event - DeepSeek suddenly became popular on the entire Internet. It triggered exponential user growth. Faced with the surge in access pressure, the platform server experienced service interruptions many times, and the "request failed" prompt interface became a high-frequency discussion topic among users, which set off a wave of self-built AI private deployment.

After more than two months of experience, I have learned how to build a Deepseek environment from a novice, and how to implement the RAG (enterprise and individual) knowledge base from theory to technology. I have also summarized my experience in the process of knowledge application transformation.

A. Knowledge base route selection.

1. If you use AI as a personal auxiliary tool and don’t want to mess around with it, and if sensitive information is not involved, it is recommended to use online AI tools on the Internet directly. The product is ready to use out of the box, simple and convenient, and focused on application.

a. Install the open source and free knowledge base application locally, then apply for the API from the platform and purchase tokens.

b. Use AI knowledge base applications developed by major Internet companies to implement personal AI knowledge base applications. For example: Zhihu knowledge base, Doubao local knowledge base.

2. If sensitive information transactions are involved ( for example, contract review, financial analysis, data analysis, knowledge base, etc.), private deployment is recommended.

[Summary]: Ordinary individual users tend to prefer Doubao local solutions + Internet AI applications; organizations or families with multiple members and advanced individual users tend to prefer local knowledge base construction + Internet AI applications.

B. Misunderstandings in the application of knowledge base.

1. They believe that the privately deployed local knowledge base (large model) can directly obtain information from the Internet and use the knowledge base as Baidu.

2. Putting hope on the "intelligence" of AI, believing that as long as the relevant data is directly imported without any sorting, the desired results can be obtained with the help of the AI ​​big model. This is exactly the beginning of the magic of the AI ​​big model.

3. To make the knowledge base smarter, human intervention is still needed. First, the data source must be manually sorted (such as regular typesetting, removal of invalid formats and content, and removal of erroneous information) to obtain a clean data source.

C. Applicable scenarios of large knowledge base models.

1. Upload all the same type of information, and then quickly get relevant answers through conversation.

2. Through a large amount of similar information, use the AI ​​big model to perform reasoning and analysis on a certain problem (or matter) to help (or train) yourself to build a comprehensive and systematic thinking model.

3. Continuously update knowledge and sort out experience through the knowledge base, forming a virtuous closed loop of [extraction-learning-practice-summary] to build the deepest moat for individuals in the workplace.

D. Focus on content construction.

1. The core of the quality of information feedback of the knowledge base comes from the current knowledge "feeding".

2. The content of the knowledge base needs to be continuously organized. The summary of knowledge and experience will change over time, forcing yourself to constantly update and master knowledge and experience.

3. Regarding the organization of data, AI tools can be used to improve efficiency. For example, Doubao can be used to analyze, organize and purify the data first , and then the data can be directly converted into PDF or text format.

4. Data sources are divided into two categories and stored separately. One is the original data, and the other is the data converted by AI tools, which are then imported into the knowledge base. Retaining the original data is to meet the needs of future continuous refining of information.

E. Suitability principle.

1. If it does not answer the question, or is simply nonsense, the problem may be:

a. The two large models are not well adapted or the large model parameters are not configured; (as shown below, the parameter configuration will be specifically introduced later)

b. Data vector parsing issues (e.g. graphic materials in PDF documents, data tables with merged cells in EXCEL or PPT, currently cannot be parsed correctly);

c. The prompt words are not used properly. (We will introduce how to use prompt words when we have the chance. Please support and encourage us.)

2. Currently, AI technology is developing too fast. New models appear every once in a while. Be patient and wait for their evolution . Regularly adapt and optimize the parameter configuration of large models. If the effect is satisfactory, determine the optimization plan and do not change it at will.

3. The vector model and large language model of the knowledge base do not need to adopt the highest model version (with high investment cost) to adapt to current devices and just meet their own knowledge application needs.

F. Input-output ratio, calculate the efficiency.

(A high budget is good enough, a limited budget is good enough; the risk of sunk costs of failure.)

1. Although there are AI large models that are claimed to be able to run without a dedicated graphics card, in actual experience, it can only be said that they can be used (the large model has slow running feedback), the effect is not ideal and there is no feasibility.

2. At present, large models still consume a lot of hardware performance. Considering the cost-effectiveness of investment, the following computer configurations are given priority:

a. A discrete graphics card with more than 12G video memory (pay special attention to the video memory capacity, which is the prerequisite for smooth use);

b. Two memory sticks with 32GB or more (this is the prerequisite for whether you can use a higher-level large model and run in mixed mode under the existing configuration);

c. 512GB SSD solid state disk (purely for fast loading);

d. Match a power supply that is slightly higher than the power requirement of the graphics card. (If the voltage and current are stable, all devices on the motherboard will be stable)

G. Configuration reference.

The following is a configuration plan for building a personal knowledge base project and upgrading the core components based on the original host (B450 motherboard + AMD 5600G + 512G SSD).

---【End】---