What is the AI ​​knowledge base? How to adjust the parameters for different application scenarios?

If you find that AI often "talks nonsense in a serious manner", especially for serious scenarios: internal corporate work, scientific research projects, etc., there is no way to directly use the results of the large model.

At this point, you need to use the "knowledge base", which can allow AI to output results that better meet your needs.

I have read a lot of AI knowledge base articles on the Internet. They are all very complicated and talk about principles. But for most of us, we don’t need to know too many principles. Instead, we need to be “fed” and told what to do.

So today I will "feed you a lesson". First, I will briefly explain the principle of the knowledge base and what it contains. Finally, I will give you how to configure the specific parameter solutions for three application scenarios (content creation, enterprise knowledge management, and data analysis and scientific research ). You can copy the homework directly.

1. What is AI knowledge base?

AI knowledge base, in simple terms, is a "small warehouse" that "feeds" professional knowledge to AI models  . We know that although large language models like ChatGPT have learned a large amount of Internet information, they may not be accurate enough in terms of professional knowledge in specific fields, and may even "talk nonsense seriously."

Let's take a simple example: If you ask AI "How was my company's sales performance last quarter?", ordinary AI models cannot answer it at all because it does not have your company's internal data. But if you build an AI knowledge base that contains your company's sales data, AI can give an accurate answer.

The core function of the knowledge base is to enable AI to answer questions based on the real and professional knowledge you provide instead of "imagining" them out of thin air.

2. How does the AI ​​knowledge base work?

Now that we understand what an AI knowledge base is, let’s take a look at how it “magically” makes AI answers more accurate.

The working principle of AI knowledge base is professionally called RAG (Retrieval-Augmented Generation) . The whole process can be divided into three key steps:

1. Retrieval

When you ask an AI a question, the system will search your knowledge base for the most relevant content to your question. This is just like when you ask a friend a question, your friend will first recall the relevant knowledge in his mind.

2. Augmentation

The system combines the retrieved relevant information with your question to form a new, richer "prompt word." It's like giving a friend some background information to help him think.

3. Generation

The AI ​​model generates the final answer based on this enhanced prompt word. With the support of professional information in the knowledge base, the answer will be more accurate and professional.

There are three core technical models involved in this process: index model (vector model)  , reranker model  and language model  .

The indexing model is responsible for converting text into a vector form that computers can understand, facilitating quick retrieval; the rearrangement model  accurately sorts the results of the preliminary search to ensure that the most relevant content is used first; and the language model  is responsible for generating natural and fluent answers based on the search results.

These three models work together to support the powerful functions of the knowledge base.

3. What parameter configurations are needed to build an AI knowledge base?

For non-technical personnel, understanding some basic parameter configurations can help you better use the AI ​​knowledge base.

Recently I am deploying FastGPT (the most powerful knowledge base tool, beating Coze and dify). The following are the key parameters I see when using FastGPT:

1. Index model (vector model) configuration

• Model ID : The type of vector model used, such as text-embedding-3-small, text-embedding-v2, etc. It is equivalent to selecting different types of "memory assistants", each with its own expertise.
• Normalization : This is a mathematical processing method. Simply put, it adjusts all data to a unified standard. Just like adjusting rulers of different lengths to the same length, it is easier to compare which one is longer. After turning it on, it can improve the accuracy of retrieval.
• Default chunk length : How big the knowledge is split into, common values ​​are 512 or 1024. Imagine splitting a thick book into small cards, each card contains a certain amount of text. The smaller the chunk, the more accurate the search, but the context may be lacking; the larger the chunk, the more complete the information contained, but the search accuracy may decrease.
• Max Context : The maximum amount of text that can be processed at one time, such as 8,000 characters. This is like how much content the AI ​​can remember at one time. The larger the value, the more text the AI ​​can process at one time, but it will also consume more resources.
• Enable re-ranking : If enabled, the re-ranking of search results can be optimized. Just like a librarian can not only find relevant books, but also rank them according to their relevance to your needs.

The better index models on the market are:

• BGE-M3 : supports multiple languages, long text (8K tokens), suitable for knowledge bases that require high-precision semantic retrieval
• OpenAI text-embedding-3-large : Performance benchmark among closed-source models, supports multimodal embedding, and has convenient API integration

2. Reranker Configuration

The re-ranking model is an indispensable part of the knowledge base system, which can greatly improve the relevance of the search results. The role of the re-ranking model is to "secondarily screen" the preliminary results of the vector search to ensure that the content finally passed to the large language model is truly relevant to the user's question.

Common configuration parameters include:

• Model selection : Select the appropriate reordering model based on language and scenario requirements
• Relevance threshold : Set the minimum score for result relevance
• Maximum number of reorders : The number of search results processed at one time, usually 10-50

The excellent rearrangement models currently on the market are:

• Jina Reranker v2 : supports more than 100 languages, has low latency, and is suitable for multilingual knowledge bases
• BGE Re-Ranker v2 : Top performance in Chinese and English, lightweight design, suitable for Chinese knowledge base
• Cohere Rerank 3 : Business model, supports long context and multi-structured data, suitable for enterprise-level applications

3. Language model configuration

• Model suppliers : such as OpenAI, Baidu Wenxin, Zhipu, etc. It is equivalent to choosing different brands of "brains".
• Model type : Select a specific model, such as GPT-4o, Qwen1.5, etc. Different models of "brains" have different specialties and performance.
• Maximum context : The maximum length of text that the model can handle, such as 32,000 characters. This determines how much content the AI ​​can "see" at a time. If your document is very long, you need to choose a model that supports a larger context.
• Maximum citations from the knowledge base : The maximum amount of content that the AI ​​can quote from the knowledge base when answering questions. This is like how much content the AI ​​can extract from a reference book to answer your question.
• Maximum response tokens : Controls the length of the answer, such as 8000. This is equivalent to limiting the length of the AI's answer to prevent the answer from being too lengthy.
• Maximum temperature : controls the creativity of the answer. The lower the value, the more conservative and accurate it is. The higher the value, the more creative it is. When the temperature is low, the AI ​​is as rigorous as a textbook. When the temperature is high, the AI ​​will be more imaginative.
• Display Top-p parameter : controls the randomness of AI word selection. Simply put, the higher this value, the more diverse the AI's answers are; the lower the value, the more fixed the answers are.
• Show stop sequence parameters : Set the conditions for when the AI ​​stops generating responses. Like a signal telling the AI ​​"I'm done with this."
• Support tool calls : Allow AI to use various "tools" in the answering process, such as calculators, search engines, etc. Just like humans use various auxiliary tools when solving problems.
• Support function calls : Allow AI to call specific program functions. For example, AI can help you send emails, query databases, etc.
• Support image recognition : AI can understand and analyze the content of images. For example, if you upload a product image, AI can recognize it and answer related questions.
• Support output thinking : AI will show its thinking process, not just the final conclusion, just like students showing the steps to solve a problem.
• Used for knowledge base file processing : AI can process and understand various file formats uploaded to the knowledge base.
• Used for question classification : AI can determine the type of question and adopt different answering strategies based on the type.
• For text extraction : AI can extract key information from complex documents. It is like an efficient article summary tool.
• For tool call nodes : In the workflow, AI can automatically call the right tool when needed. Imagine a robot on a production line, using different tools as needed.

Current language models that perform well include:

• Claude 3.7 Sonnet : Designed for reasoning, suitable for fields that require deep logical reasoning
• GPT-4o : strong multimodal input capability, fast response speed, suitable for multimodal knowledge base
• DeepSeek-R1 : Performance benchmark among open source models, fast inference speed, low cost
• Llama 3.3 : Fully open source, suitable for enterprise applications that require high customization
• Qwen2.5-Max : Excellent mathematical reasoning and programming skills, suitable for technical document generation

4. Other important parameters

• Segmentation mode : How to split long text into small chunks
• Direct segmentation : simply segment by word count, just like dividing a book into a fixed number of lines per page
• Enhanced processing : Increase semantic richness by generating questions and summaries, similar to adding subheadings and extracting key points to each paragraph
• Question-answer splitting : Split the content into question-answer pairs, just like turning the content into a one-question-one-answer interview question
• Search strategy :
• Vector search : Based on semantic similarity search, it can understand the actual meaning of the question rather than just the keywords.
• Full-text search : based on keyword matching, just like using Ctrl+F to search for specific words in a document
• Hybrid search : combines the advantages of both methods, looking at both keywords and semantics

For beginners, the default parameters can be used in most cases. As you gain experience, you can gradually adjust these parameters to achieve better results.

4. AI knowledge base application solutions in different scenarios

Different industries and demand scenarios have different requirements for knowledge bases. Below we introduce complete solutions for three typical scenarios in detail to help you choose the most appropriate configuration based on your actual needs.

1. Content creation scenarios (writing for official accounts and Xiaohongshu)

Application requirements : Content creators need AI to help write creative, interesting articles, short video scripts or graphic content that conforms to the platform style, while ensuring the accuracy and timeliness of the content.

Suggested solution :

• Model combination :
• Language model: GPT-4o (strong creativity)
• Index model: text-embedding-3-small (lightweight but moderately accurate)
• Reranking model: Jina Reranker v2 (multi-language support, fast speed)

• Detailed parameter configuration
•     Language model parameters:
• Maximum contexts: 8000 (enough for reference and creative needs)
• Maximum number of citations in the knowledge base: 3,000 (use the right amount of citations to avoid limiting creativity)
• Maximum response tokens: 4000 (to ensure sufficient content is generated)
• Temperature: 0.7-0.9 (tends to be creative, but not too scattered)
• Top-p parameter: 0.9 (maintains high vocabulary diversity)
• Support image recognition: On (convenient for reference of image materials)
• For text extraction: On (effectively extract the essence of reference content)
• Rearrange model parameters:
• Relevance threshold: 0.6 (medium standard, retaining more creative inspiration)
• Maximum number of reorderings: 30 (retain more candidate paragraphs)
• Index model parameters :
• Block length: 256 (smaller blocks are easier to extract creative elements)
• Normalization: Enable (to improve retrieval accuracy)
• Max contexts: 4000 (medium size, enough for creative content)
• Segmentation mode : Enhanced processing (enriching semantic information by generating relevant questions and summaries)
• Search strategy : vector search as the main method, supplemented by full-text search (focusing on semantic understanding)

2. Enterprise internal knowledge management scenario

Application requirements : Enterprises need AI to assist in the management and query of internal documents, process manuals, product information, meeting minutes and other information. The answers must be accurate, comply with company regulations, and be able to handle a variety of document formats.

Suggested solution :

• Model combination :
• Language model: Claude 3.7 Sonnet (strong reasoning ability, rigorous logic, but expensive haha)
• Index model: BGE-M3 (high-precision semantic understanding, multi-language support)
• Re-ranking model: BGE Re-Ranker v2 (accurate re-ranking, lightweight)
• Detailed parameter configuration
• Language model parameters :
• Maximum context: 32,000 (large capacity, suitable for complex enterprise documents)
• Maximum number of citations in the knowledge base: 16,000 (making full use of internal knowledge)
• Maximum response tokens: 3000 (moderate length, focused)
• Temperature: 0.3-0.5 (low temperature to ensure rigorous and accurate answers)
• Top-p parameter: 0.7 (moderate control of randomness)
• Support tool calling: Enable (for easy connection to internal enterprise systems)
• Support function call: Enable (can interact with enterprise database)
• For knowledge base document processing: On (processing various enterprise documents)
• For question classification: On (intelligent diversion of different types of consultations)
  
• Rearrange model parameters :
• Relevance threshold: 0.75 (higher standard to ensure accurate answers)
• Maximum number of reorders: 20 (reasonable number to ensure comprehensive information)
• Index model parameters :
• Chunk length: 512 (balance between detail and retrieval accuracy)
• Normalization: On (to improve the accuracy of enterprise terminology retrieval)
• Maximum contexts: 8000 (enough to handle most enterprise documents)
• Segmentation mode : Direct segmentation (accurately retain the original enterprise information)
• Search strategy : Hybrid search (combining semantics and keywords, comprehensive and accurate)

3. Professional field scenarios (data analysis, industry research, academic papers)

Application requirements : Researchers and analysts need AI to assist in processing and interpreting complex professional data, research reports, and academic papers. The answers must be in-depth, accurate, and professional, and able to handle complex academic concepts and data relationships.

Suggested solution :

• Model combination :
• Language model: DeepSeek-R1 (excellent reasoning ability) + Qwen2.5-Max (strong mathematical and logical ability)
• Index model: OpenAI text-embedding-3-large (high-dimensional semantic expression)
• Reranking model: Cohere Rerank 3 (good support for complex structured data)
• Detailed parameter configuration
• Language model parameters :
• Maximum context: 128,000 (very large capacity, suitable for long paper research)
• Maximum citations in the knowledge base: 32,000 (full citation of professional materials)
• Maximum response tokens: 8000 (long article, detailed analysis)
• Temperature: 0.1-0.3 (extremely low temperature to ensure rigor)
• Top-p parameter: 0.5 (limit randomness and ensure logical coherence)
• Show stop sequence parameters: On (precise control of answer format)
• Support tool calling: On (connect professional analysis tools)
• Support output thinking: On (show reasoning process)
• For text extraction: On (accurate extraction research points)
• Rearrange model parameters :
• Relevance threshold: 0.85 (high standard, ensuring accurate citation content)
• Maximum number of reorderings: 50 (large number of candidates, comprehensive coverage of relevant knowledge points)
• Index model parameters :
• Block length: 1024 (larger blocks retain complete professional concepts)
• Normalization: On (to improve the accuracy of professional terminology retrieval)
• Maximum contexts: 16,000 (large capacity for handling academic content)
• Segmentation mode : Question and answer splitting + direct segmentation (while ensuring professional accuracy and question targeting)
• Search strategy : vector search + full-text search (taking into account both semantics and professional terminology)