What is GraphRAG?

GraphRAG (Graph Retrieval-Augmented Generation) is an advanced RAG technique that structures knowledge from source documents as a graph of entities and relationships. RAG is a vector-based machine learning technique to extend a large language model’s knowledge without retraining. GraphRAG’s graph modeling approach uses graph traversal for information retrieval, an alternative to RAG’s semantic search. GraphRAG outperforms the baseline RAG on highly interconnected data and aggregate queries.

Why is GraphRAG important?

GraphRAG addresses the limitations of retrieval-augmented generation (RAG) when synthesizing from private datasets. RAG, or baseline RAG, works by combining semantic search with a vector database that a large language model (LLM) uses as a reference. RAG allows LLMs to answer queries beyond their trained knowledge, with outside information. When an LLM receives a query, the baseline RAG runs a vector search that returns semantically relevant chunks from the private database. This allows the LLM to understand proprietary or domain-specific data, such as product price.

However, RAG isn't designed for surfacing connections in data across separate documents. While RAG excels with semantic similarity search in single documents, it struggles in complex queries that require relational reasoning across many documents. Consequently, the LLM might fail to provide contextually rich responses to multi-hop, relational, and hierarchical queries. For example, if you query a baseline RAG, “What is the price of product A?”, it returns “Price of product A is $10”. However, it couldn’t confidently answer questions like “Why is product A more expensive than product B?”

GraphRAG is designed to improve the explainability and reasoning capability of baseline RAG. Instead of focusing on semantic search within single documents, GraphRAG maps entities across all documents in a knowledge graph. By connecting related entities across documents, GraphRAG enables the LLM to answer more holistically on topics that a vector-based retriever couldn’t. Additionally, GraphRAG improves explainability, so you can more easily understand why specific entities are mentioned in the generated response.

How does GraphRAG work?

GraphRAG with Amazon Neptune Graph

GraphRAG combines LLMs with a graph database for complex reasoning across entities to provide contextually relevant responses. Instead of retrieving data directly from a private vector database, it introduces a knowledge graph to augment the query. By using techniques such as graph traversal and depth-first search, the retrieval module maps, analyzes, and identifies nodes in the knowledge graph.

Here are the steps in GraphRAG:

The user queries the LLM
The LLM extracts entities and relationships from the query and maps entities to nodes and relationships to edges
The GraphRAG uses a graph query language to retrieve specific information from the knowledge graph
Once retrieved, the graph-based structure is filtered to remove irrelevant nodes
The LLM synthesizes the answer based on the retrieved graph and the original query

The actual implementation of GraphRAG requires indexing the knowledge graph before querying can occur.

Indexing phase

The indexing phase creates the knowledge graph from private documents. To do so, the LLM uses the named entity recognition (NER) natural language processing technique to identify key entities from multiple input datasets.

Consider an organization connecting an LLM to data warehouses from different business departments. During knowledge graph creation, the retriever extracts key information such as products, pricing, and dates from marketing, sales, and customer support from different databases. Then, the GraphRAG system discovers and establishes relationships amongst the extracted entities. It uses semantic and co-occurrence relationships to connect entities.

The goal of indexing is to cluster related information together. Here, GraphRAG applies machine learning techniques like Leiden to discover entities or nodes with strong contextual relationships. We call these groups of nodes a community. Often, a knowledge graph consists of several community structures that are loosely connected. For each community of nodes, the LLM generates a top-level summary, which helps identify overarching themes during inference. These graph-structured data points make it easier for LLMs to retrieve highly relevant information for complex reasoning.

Finally, the GraphRAG stores the knowledge graph. In LLM systems that support hybrid retrieval, the retriever generates vector embeddings and stores them in a vector store alongside the graph database.

Query phase

Querying involves users actively prompting the LLM for domain-specific information. When the LLM receives a query, it identifies and extracts entities mentioned in it using techniques such as named entity recognition. Then, the LLM connects the entities so it can interpret their relationships more effectively.

GraphRAG uses graph query languages, such as Cypher, Gremlin, and SPARQL, to traverse the graph and map entities to nodes and edges. This allows the GraphRAG system to map the connected entities to the underlying knowledge graph. Once identified, the GraphRAG system returns the associated entities, which are then preprocessed with scoring. Lastly, the system assesses the relevance of retrieved nodes and chooses the most relevant ones to generate the answer.

Local vs. global search

Depending on the type of query, the GraphRAG system may conduct a local or global search. Local search uses structured data from the knowledge graph and unstructured data in individual documents to form a contextually accurate answer. Usually, it answers questions like “What are the contract terms for business X?” Meanwhile, global search combines the summaries of multiple communities to provide a holistic response. Global search improves LLM accuracy when answering questions like “What are common features across our business contracts?”

What is a knowledge graph?

Knowledge graph for contextual reasoning

A knowledge graph consists of entities that are interlinked and arranged in a complex network. Entities are self-contained objects, such as people, places, or events. Different nodes are connected by an edge. In GraphRAG, the knowledge graph captures relationships between entities that vector embeddings cannot represent.

Think of a knowledge graph as a large web where multiple objects are connected to each other. For example, in the sentence ‘Mary bought a house’, Mary and house are nodes while bought is the edge. Unlike traditional databases with rigid schemas, a knowledge graph serves as a flexible data modeling technique to capture, represent, and interpret relational structures. ML engineers build a knowledge graph by using LLM extraction, NER, and relationship extraction techniques.

GraphRAG vs. baseline RAG

Generally, a baseline RAG is easier to set up than a GraphRAG because it is simpler. Baseline RAG stores chunks of information as vectors in a vector database. When retrieving information, the baseline RAG extracts from a vector database. Meanwhile, GraphRAG turns private datasets into knowledge graphs that capture the contextual relationships between entities. Because it uses a different retrieval process, GraphRAG outperforms the baseline RAG on complex reasoning and holistic summarization tasks.

Baseline RAG performs well in vector-based similarity search, which works well for simple lookups. GraphRAG relies on graph schema and relationship extraction in response generation, which takes longer for simple searches. When a baseline RAG retrieves entities, it chooses those that semantically match the query. Conversely, a GraphRAG system assembles entities with relevant context when generating answers. Baseline RAGs can sometimes hallucinate when they can’t determine contextual relationships. On the other hand, GraphRAG traces the knowledge graph to provide accurate responses. Because GraphRAG retrieves graph-based entities, you can explain why the LLM chooses specific entities in its response more easily.

What are the types of GraphRAG patterns?

Below, we share different ways to implement GraphRAG.

Graph-only retrieval

Graph-only retrieval relies solely on traversing the graph structure that the LLM built for retrieval. Instead of using semantic search to match relevant embeddings, it navigates connections between relevant nodes with traversal algorithms like depth-first search and adaptive retrieval. You can also use graph neural networks (GNNs) for graph traversal with advanced reasoning.

Vector + graph hybrid

GraphRAG excels at interpreting relationships between entities. However, graph-only retrieval lacks semantic search matching. Combining vector embeddings with GraphRAG allows organizations to overcome the limitations of both retrieval models. In this pattern, the graph connects multiple nodes storing vectorized data points. Sometimes known as hybridRAG, this model performs optimally for complex, multi-hop questions like “Which shoppers showed behavior similar to John, and what strategies led to successful conversions?”

Agentic GraphRAG

Agentic GraphRAG uses agentic AI to perform graph-based retrieval and generation. Agentic AI is a collective group of AI automated software that work cohesively towards a common goal, with independent contextual decisions. By integrating agentic AI with GraphRAG, the system can query, analyze, and improve the response through autonomous multi-step reasoning.

What are the use cases for GraphRAG?

Organizations can use GraphRAG to improve information retrieval and query responses in these areas.

Enterprise knowledge management

GraphRAG can enhance knowledge management workflows with cross-document question answering. GraphRAG surfaces connections between disparate documents, policies, and databases to provide employees contextually relevant answers. For example, an ecommerce chatbot can retrieve past sales transactions, specific customer information, and product features when generating an answer.

Cybersecurity

Security teams can integrate GraphRAG with their cyber defense infrastructure to identify suspicious patterns. Graph-based traversal allows security systems to correlate isolated alerts to identify meaningful patterns. By mapping the structural relationships among threat actors, vulnerabilities, assets, and prior incidents, they can improve efforts to anticipate future data risks.

Healthcare and life sciences

Drug discovery involves extensive effort to analyze compounds, mechanisms, clinical trials, and patient outcomes across large volumes of datasets. Medical researchers can augment their efforts with GraphRAG to generate evidence-based insights from a wide variety of sources. Additionally, healthcare professionals can analyze patient diagnoses, treatment plans, and symptoms using a graph-based structure to identify potential correlations.

Financial services

Banks, fintech, and lending institutions can integrate GraphRAG with their systems to improve functions such as credit scoring and fraud detection. By leveraging graph-based connections, finance teams can identify suspicious patterns across transactions, accounts, and banking networks.

Legal and compliance

Graph-based analysis improves workflows that support cross-referencing regulations, compliance, and legal obligations in practices. Lawyers, contract clerks, and legal professionals can increase efficiency and conciseness when analyzing large document sets.

What are the key considerations for GraphRAG?

GraphRAG introduces contextual understanding that addresses baseline vector RAG’s limitations. However, challenges remain for scaling the adoption of GraphRAG across industries.

Indexing cost and latency

Compared to a baseline RAG, a GraphRAG uses an LLM to index and map entities based on their association. This process is resource-intensive and costly. Therefore, some applications, especially those that require simple question-and-answer lookups, don’t justify the cost of a GraphRAG’s setup.

Graph maintenance

As organizations grow, they must ingest new documents to keep the knowledge graph up to date. However, increasing the graph size can degrade retrieval quality as the dataset grows to a very large scale. If the graph data grows over a certain limit, the retriever struggles with noise and multi-hop reasoning. To overcome this phenomenon, ML teams combine community detection with individual node retrieval.

Retrieval strategy selection

Depending on the query type, some variants of GraphRAG perform better than others. For example, graph-based retrievers with strong edge weighting and vector-indexed nodes perform better at factual retrieval. Meanwhile, abstract queries that require cross-page reasoning require community summaries and relationship-rich graphs.

Evaluation

When evaluating GraphRAG’s performance, choose frameworks that are well-suited to graph-based retrieval. GraphRAG systems retrieve information as subgraphs of entities. Subgraphs are typically serialized into text so LLMs can process them. As such, standard RAG metrics, which evaluate precision and recall at the chunk level, may not be entirely sufficient. Instead, use methods beyond token-based metrics, such as multi-hop reasoning benchmarks and answer faithfulness metrics.

How can AWS support your GraphRAG requirements?

AWS offers managed and self-hosted options for building GraphRAG applications, combining graph infrastructure with foundation model access. Here are some foundational GraphRAG services:

Amazon Bedrock Knowledge Bases is a fully managed RAG capability with in-built session context management and source attribution that supports GraphRAG. If you choose Amazon Neptune Analytics as a vector store, Amazon Bedrock Knowledge Bases automatically creates embeddings and graphs that link related content across your data sources. Bedrock Knowledge Bases leverages these content relationships with GraphRAG to improve the accuracy of retrieval, enabling more comprehensive, relevant, and explainable responses to end users.

Amazon Neptune is a serverless graph database service for connected data and improved AI accuracy. To make AI application development easier, Amazon Neptune Analytics offers fully managed GraphRAG with Amazon Bedrock Knowledge Bases, and integrations with Strands AI Agents SDK and popular agentic memory tools.

Get started with GraphRAG on AWS by creating a free account today.

What Is GraphRAG?