Knowledge graphs are revitalized, inspiring deep reasoning in large models - Yesterday, Today and Tomorrow

Written by

Iris Vance

Updated on:June-19th-2025

“What is innovation? Innovation is connecting previously unrelated knowledge together.”
(This is a revision of the original sentence and may not be accurate)
A closed-door meeting

One night two months ago, the brightly lit Zhipu AI headquarters organized a closed-door meeting for vertical industries. Industry academic experts and big model solution experts discussed the prospects for the implementation of AI. When discussing the topics of "big models" and "innovation" , the above remarks left a deep impression on me.

The reality is indeed so: the combination of "car" and "renewable energy" gave birth to "new energy car" , and the combination of "wearable device" and "sensor" gave birth to "electronic bracelet" .

From a micro perspective, the big model uses Beam Search to string together originally unrelated "words" to form a sentence that humans can understand.

Both are “connected” in essence , one macro and one micro.

Maybe there is a "certain" relationship between them (cars -> gasoline -> non-renewable energy -> energy -> renewable energy) , but no one has broken the window paper before. Then the first person who breaks it directly is called "innovation" .

Knowledge Graph and the Human Brain

When we ask machines to do thinking and reasoning similar to the human brain, we usually think of "knowledge graphs". Let me draw a picture of the new energy vehicle example mentioned above:

People often say "car" and "gasoline" together, and the brain hardly needs to think, just blurt out the words. This is "fast thinking". However, if you want to associate "car" with "renewable energy", you need to follow the diagram above. An inventor may need 3 seconds, and an ordinary person may never complete it in his entire life. This is "slow thinking".

However, "fast" and "slow" are constantly evolving, and for us earthlings in 2024, [car->renewable energy] has become as easy to say as [car->gasoline].

Knowledge Graph——Danger! ?

The key to making the machine realize the above example lies in two points:

1. How to build this “map”? (Map construction)

2. How to use the established “graph”? (Graph Reasoning)

In the past, there were a lot of studies on solving these two problems. Now, it seems that they are no longer valid. For example:

To build a knowledge graph , we used to have to do SPO triple extraction (named entity recognition/relation extraction), knowledge fusion, etc., but now we can use a prompt to replace all the tasks;
In the past, when using graphs for reasoning , we had to consider representation learning reasoning and symbolic logic reasoning. Now that large models have absorbed knowledge from all over the world, the graph above may be directly internalized into model parameters and directly generate answers. If "fast thinking" is not effective, let CoT simulate people's "slow thinking" to greatly improve the effect.

Is the knowledge graph still useful? Or should we think about how to redefine the knowledge graph in the era of big models? (Leave it as a question for some time to think about in the future)

Old tree sprouts new leaves

With this question in mind, I started some scattered explorations and discoveries: It turns out that along the CoT route, many reasoning solutions based on knowledge graphs have emerged.

For example, GoT (Graph of Thoughts), ToG (Think on Graph), RoG (Reason on Graph), etc. Taking a graph in ToG as an example, the effects of the three technical solutions are compared:

From "THINK-ON-GRAPH: DEEP AND RESPONSIBLE REASONING OF LARGE LANGUAGE MODEL ON KNOWLEDGE GRAPH"

The question posed in the picture is: "What is the majority party in the country where Canberra is currently located?"

Option 1: Throw it directly to the big model, it will say, my knowledge stops at September 2021, when the majority party is the Liberal Party (obviously wrong)
Solution 2: The traditional solution of using knowledge graph to do KBQA. The big model first finds that the "country" where "Canberra" is located is "Australia". Then, when it looks for the "majority party" based on "Australia", it finds that this piece of knowledge does not exist (not exit) , and then replies, sorry, I don’t have enough information (obviously it’s not that there is not enough information, but that I’m too lazy)
Option three: This is the option proposed by the author. Even if there is no clear "explicit relationship" to express the "majority party", we can still infer based on"Australia->Prime Minister->Anthony->Leading Party->Labor Party"that the majority party is most likely the "Labor Party".

This reasoning process is very similar to the reasoning process of the human brain.

The future is now

The knowledge graph returns to its essence - the association of knowledge .With the support of symbolism , the connectionist model, which currently does not perform well in some reasoning problems, has become more powerful.

I anticipate that it will help humans connect countless high-value "points" that have never been connected before , allowing the ability to discover knowledge and innovate to reach a whole new level.

The direction of extension

The above is all about knowledge graphs. If we abstract a little more and come to the "graph" structure, it will be a broader field. Regarding how "graphs" in a broader sense can be combined with large models to generate greater value, the following studies are listed first. Friends who are interested are welcome to discuss with us:

A Survey of Graph Meets Large Language Model: Progress and Future Directions
Graph Meets LLMs: Towards Large Graph Models
GPT4Graph: Can Large Language Models Understand Graph Structured Data? An Empirical Evaluation and Benchmarking
GRAPHLLM: BOOSTING GRAPH REASONING ABILITY OF LARGE LANGUAGE MODEL
...... (I won’t list them all, here is the gitHub collection of the paper, https://github.com/XiaoxinHe/Awesome-Graph-LLM )

Of course, there is also a review specifically on the combination of "knowledge graph" and large models: "Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications"

From " Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications "

In addition to academic aspects, I have recently noticed that LangChain has released a new Agent architecture - LangGraph (https://blog.langchain.dev/langgraph/), which applies graph structure to the orchestration of Agent intelligent bodies. At the same time, it provides an answer sheet on architectural design in the direction of Multi Agent. Friends who are interested in this aspect are also welcome to communicate.