Common pitfalls of RAG: How to make AI cheat effectively in exams
In-depth interpretation of the misunderstandings and coping strategies of building enterprise knowledge bases, and the key blind spots of RAG technology in practical applications.
Core content:
1. The working principle of RAG technology and its application in enterprise knowledge bases
2. Common misconceptions and their impact on the generation of answers by large models
3. Three major pitfalls in building enterprise knowledge bases and solutions
DeepSeek was very popular during the Spring Festival. I don’t know why it also made personal AI knowledge base popular.
In this article, we will not talk about the kind of toys that “just drag the article in and ask the AI” because no one will listen even if we talk about it.
Let’s talk about the construction and application of the “Enterprise Knowledge Base ,” or RAG, a common blind spot in this work.
How RAG works
The three letters of RAG stand for Retrieval-Enhancement-Generation . Its principle is actually very simple, that is, the literal meanings of three words are done in order:
1. When users ask questions, they don’t send them to the big model, but use the questions to [search] for relevant information 2. Splice the retrieved data with the original question to [enhance] the quality of the prompt words sent to the big model later (add context) 3. After receiving the "question" and "information", the big model generates answers to answer the user
Note that there are two very important understandings here:
1. Regarding the wording, I used “information” instead of “answer” in the second step; 2. The big model plays the role of "editor" in this process, not "customer service" or someone who creates answers
Most of the poorly done RAGs are because the data retrieved is assumed to be the answer.
This wrong perception will cause you to lack the judgment reminders for possible errors or even completely wrong data in the prompt words that constrain the large model to generate answers in the last step.
Incorrect data can cause large models to produce "hallucinations".
Let's take an example to deepen our understanding:
You didn't attend class or review before the exam, and when you encountered a question you didn't know how to answer, you asked your deskmate for the answer. But your deskmate was very busy and just gave you a few cheat sheets.
Since you didn't know anything, you just edited the words in the cheat sheet and finished the test.
Once you understand the above, you will most likely be able to avoid the following pitfalls.
The article is better to be longer so that it contains more useful information. I will continue to use the example of cheating in an exam to explain the following pitfalls to you.
Pitfall 1: Direct insertion
The knowledge materials within your company are cheat sheets for the big model.
If you just tear up the textbooks and put them in your pockets before bringing them into the exam room, your final grade will definitely not be very good.
It’s not that it’s inconvenient for you to find the answer. The biggest problem here is that when you are using the textbook, you may tear a related piece of knowledge into two halves!
During the exam, you can only take out three separate answers at a time, which means you will probably only get half of the answers and two completely irrelevant answers.
You can imagine what answer you would come up with.
Therefore, it is totally ineffective to simply cram the information into the knowledge base and simply divide it into sections based on character length or paragraph breaks.
Cleaning and manual splitting are compulsory courses for enterprise knowledge base.
"Can I divide a book into sections instead of tearing it apart?"
OK, but remember to apply for more token budget.
Pitfall 2: Acting as customer service
I have seen many RAG system prompts where the first sentence is "You are a professional customer service ..."
The big model’s identity recognition of the customer service role is “answering customers’ questions”, but as mentioned earlier, the big model plays the role of “editor” in RAG.
Its core mission is to answer questions based on the information we provide.
Since you have chosen to provide it with information, it means that its ability to answer questions can no longer satisfy you.
The identity of customer service will make the big model do its best to serve customers. When the information you provide it cannot support it to complete the task, it will use the power of "illusion" to give a decent answer.
The role of "intern who helps customer service staff organize information" would be better.
Pitfall 3: Too many demands
"A company that uses punch cards to evaluate employees will only make their employees work fewer hours."
“Strict family education cannot produce an artist.”
The same goes for large models.
I understand that everyone is worried about the big model talking nonsense, so they have formulated strict constraint prompts, similar to:
1. You must… 2. Be sure to provide... 3. Don’t imagine what doesn’t exist…
And various prayer-like ability granting such as “you are familiar with…”, “you are proficient in…”, “you are good at…” will make it more difficult for the big model to focus on answering real questions.
What a large model needs more is an understanding of the background information of the task it is responsible for and how to deal with problems it may encounter.
The following is a snippet of the system-level prompts for the teaching assistant agent I configured for a certain course:
You will play the role of a course assistant, organizing and editing the corresponding response text based on the students' questions and the relevant information I provide.
## Mission Background and Requirements
- I analyzed and refined the course transcripts and created a Q&A database
- When students ask questions, I will conduct a multi-dimensional search in the database and send you the retrieved information and the students' questions.
- Your task is to edit and polish the questions and retrieved materials to generate response text that can answer the questions
- Note: The database cannot cover all questions, so there may be cases where the information is irrelevant, incomplete or even without supporting information. This means that I did not teach the relevant content in the course. In this case, you can guide students to consult Teacher Zhang Jia instead of outputting the question response text.
- Your actual responsibility is not to answer students' questions, but to ** inform students whether there is a professor in the course and in which section the course will be taught based on the information, and guide them to learn the course **
- In order to help students understand and judge whether your response text is accurate, you need to briefly introduce how the course solves his problem based on the supporting materials, and then provide guidance for further learning
- Your reply will be sent to the WeChat group. Make sure it is concise. If there are multiple paragraphs, add blank lines to improve the reading experience.
- Ensure the reply is plain text, **Do not use any markdown identifiers**
