Talk about the four levels of RAG

Choosing the right RAG (Retrieval Augmentation Generation) architecture depends primarily on the specific use case and implementation requirements, ensuring that the system meets the mission needs.

Agentic RAGs will grow in importance, in line with the concept of Agentic X, where agent capabilities are embedded in personal assistants and workflows.

The “X” here represents the infinite adaptability of agent systems, enabling seamless task automation and intelligent decision making across diverse environments, thereby increasing organizational efficiency and autonomy. Synthesizing disparate document sources is critical to effectively resolving complex, multi-part queries.

1. Overview

The challenges of providing accurate RAG implementations include retrieving relevant data, accurately interpreting user intent , and leveraging the reasoning capabilities of LLMs to accomplish complex tasks. Reasoning capabilities can be enhanced through agent approaches to RAGs like ReAct, where reasoning about events and behavioral sequences are created.

One interesting fact I found from this study is that it points out that there is no one solution that fits all data augmentation LLM applications.

Context refers to the information surrounding a conversation that helps AI understand the user’s intent and provide relevant, coherent responses.

This includes factors such as the user's previous input, current task, environment, and any external data that may affect the conversation.

Effective contextual processing enables AI to maintain consistent and personalized conversations, tailoring responses based on the ongoing interaction and ensuring the conversation feels natural and meaningful.

User intent detection

In many cases, the root cause of poor system performance is either a failure to prioritize the task or a task that requires a combination of skills that must be carefully differentiated to achieve the best results.

Intent refers to the underlying purpose or goal behind user input and represents what the user hopes to achieve or convey through the query.

Recognizing intent allows AI systems to respond appropriately.

2. RAG data classification

Level 1: Explicit Fact Query

Ask directly about specific, known facts.

Queries are about explicit facts that exist directly in the given data and do not require any additional reasoning.

This is the simplest form of query, where the model’s task is primarily to locate and extract relevant information. When a user asks a question, the RAG implementation locates the facts contained in the chunked data.

Level 2: Implicit fact query

Finding facts indirectly requires interpretation to determine the answer.

Queries involve implicit facts in the data that are not obvious and may require a certain degree of common sense reasoning or basic logical inference.

The necessary information may be spread across multiple sections or require simple reasoning.

For example, What is the majority party now in the country where Canberra is located? This question can be answered by combining the fact that Canberra is located in Australia with information about the current majority party in Australia.

In the second level we begin to see the introduction of elements of reasoning and action, thus taking a more agentic approach to RAG.

Level 3: Explainable Principle Inquiry

Focuses on understanding the reasoning behind facts and requires data to support logical explanations.

These queries require both factual knowledge and the ability to interpret and apply domain- specific guidelines that are critical to the context of the data .

Such justification is often provided in external resources but is rarely encountered in the initial pre-training of general language models.

For example, in a financial audit, an LLM may need to follow regulatory compliance guidelines to assess whether a company's financial statements meet standards.

Similarly, in technical support, a troubleshooting workflow may need to be followed to assist users, ensuring responses are accurate and in accordance with established protocols.

Level 4: Hidden Rational Doubt

Seeking deeper insights often requires context-based reasoning to uncover underlying meanings or implications.

Such queries require AI to infer complex principles that are not explicitly documented based on patterns and outcomes observed in the data.

These hidden reasons involve implicit reasoning and logical connections that are difficult to pinpoint and extract.

For example, in IT operations, a language model might analyze patterns in past incident resolution to determine successful strategies.

Similarly, in software development, AI can draw on past debugging cases to infer effective problem-solving approaches. By synthesizing these implicit insights, the model is able to provide responses that reflect nuanced, experience-based decisions.

Explainable and hidden principles shift the focus to the ability of the RAG system to understand and apply the reasoning behind the data.

These levels require deeper cognitive processes where agent frameworks align with expert knowledge or extract insights from unstructured historical data.

Three conclusions

According to the research and considering the above diagram, there is a distinction between queries that require explicit facts and queries that rely on implicit reasoning.

For example, inquiries regarding visa eligibility require clear facts from the consular guidance (L3), while questions regarding the economic impact on the company's future development require analysis of financial reports and economic trends (L4).

Data dependency in both cases highlights the importance of external sources – be it official literature or expert analysis.

In both cases, providing reasons helps to contextualize the response, providing not only the answer but also the sound reasoning behind it.