--- title: "GraphRAG vs Vector RAG: What Each Does Best for AI Agents" url: https://syntaxradar.com/radar/stop-using-vector-dbs-for-agents-hierarchical-knowledge-graphs-vs-rag author: "Abdul Rehman, Lead Systems Architect and Technical Director" publishedAt: 2026-10-25T08:25:00.000Z dateModified: 2026-10-29T10:33:58.900Z category: "#SYSTEMS-ARCH" topic: "graphrag" primarySource: https://arxiv.org/abs/2404.16130 summary: "GraphRAG vs vector RAG explained for non-experts: how each retrieves information, where graphs help, where plain search wins, and what it costs." --- # GraphRAG vs Vector RAG: What Each Does Best for AI Agents **Short answer:** Microsoft Research's GraphRAG builds a knowledge graph and community summaries before answering. Its authors report better comprehensiveness and diversity than baseline RAG on whole-collection questions, while vector search still suits simple lookups. **Takeaway:** GraphRAG helps with questions that span a whole document collection, while plain vector search remains the simpler choice for single-passage lookups. **Key figures (as reported by the source):** - Default text unit size: 1,200 tokens - LazyGraphRAG indexing cost vs full GraphRAG (Microsoft): 0.1% - LazyGraphRAG global query cost (Microsoft): 700x lower GraphRAG is a way of feeding documents to an AI model that builds a map of the people, places and things in the text before any question is asked. Microsoft Research introduced it in a [February 2024 blog post](https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/) and a [paper on arXiv](https://arxiv.org/abs/2404.16130). It targets one weakness of ordinary retrieval-augmented generation (RAG): questions about a whole pile of documents. Vector search, the usual engine behind RAG, is still the better tool for many jobs. Which one an AI agent should use depends on the kind of question being asked. ### What is GraphRAG, and how is it different from standard RAG? Standard RAG works in three moves. Documents are cut into chunks, each chunk is turned into a list of numbers called an embedding (a numeric fingerprint of its meaning), and at question time the system fetches the chunks whose fingerprints sit closest to the question. A vector database is the store that makes this lookup fast. GraphRAG adds a preparation stage. According to the [official GraphRAG documentation](https://microsoft.github.io/graphrag/), it extracts a knowledge graph from raw text, groups the graph into a hierarchy of communities, and writes a summary for each community. A knowledge graph is simply a set of entities (a company, a person, a product) joined by labelled relationships ("supplies", "manages"). A community is a cluster of entities that are closely connected to one another in the graph, such as a supplier, the orders it ships and the people who manage them. The community summary is a short report a language model writes about that cluster, so the system holds a ready-made overview of each topic before any question arrives. The Microsoft authors say baseline RAG struggles to "connect the dots" across scattered pieces of information. It also performs poorly on questions that need a view of the whole collection, such as asking for the top themes in a dataset. For those, vector similarity returns text that looks related but does not summarise anything. ### How does GraphRAG build and search its index? The [indexing pipeline](https://microsoft.github.io/graphrag/index/default_dataflow/) has several steps. Input documents are split into text units, 1,200 tokens each by default. A language model then reads every text unit and extracts entities and relationships, and a second pass condenses the descriptions of repeated entities into one. Next, a clustering method called hierarchical Leiden community detection groups closely connected entities. A language model writes a report for each group, working from the bottom of the hierarchy upward. Entity descriptions, text units and community reports are finally embedded into a vector store, so vectors remain part of the system. Because every text unit passes through a language model, the stage that makes whole-collection questions answerable is also the stage that makes setup costly. At question time the docs list several search modes: - Global search answers whole-collection questions using the community summaries. - Local search starts from specific entities and reaches their neighbours and related concepts. - DRIFT search works like local search but adds community information. - Basic search is ordinary top-k vector search, included for questions that suit it. The hierarchy gives the system several zoom levels. Low-level communities cover narrow topics, higher ones cover broad themes, so a question about the overall picture can be answered from the broad summaries without reading every chunk. Because those summaries already exist, a whole-collection question draws on them instead of on raw text. The paper describes the global route as a map-reduce style process: each community summary produces a partial answer, and the partial answers are combined into a final response. ![A researcher pinning photographs and red string across a wall to trace connections between documents](/uploads/blog/stop-using-vector-dbs-for-agents-hierarchical-knowledge-graphs-vs-rag/gemini-supporting.jpg) ### Is GraphRAG better than vector RAG? On whole-collection questions, the authors report that it is. The paper says GraphRAG gives "substantial improvements" over a conventional RAG baseline in the comprehensiveness and diversity of generated answers. Those tests used global sensemaking questions over datasets in the range of one million tokens. The 2024 blog post tested on VIINA, a dataset of thousands of Russian and Ukrainian news articles from June 2023. An LLM grader picked a winner between the two approaches on comprehensiveness, diversity and use of supporting source material. The post says GraphRAG "consistently outperforms" the baseline, and that faithfulness (staying true to the sources) was at a similar level for both. It publishes no exact win rates. Microsoft's published tests support a direction of improvement on one specific type of question, whole-collection questions, and the judge was an AI grader rather than a human panel. Results on your own data will differ. ### When does vector search still win? Vector search is the simpler choice when the answer lives in one or two passages, because the system only has to find the right chunk and hand it to the model. Asking a company handbook for the holiday policy is the classic case: find the right chunk, quote it. The official docs keep a basic vector search mode inside GraphRAG for exactly this reason. Some third-party write-ups report that vector RAG beats graph approaches on single-document lookups and loses on questions that span many documents. One engineering blog, [tianpan.co](https://tianpan.co/blog/2026-04-09-graphrag-production-when-vector-search-hits-ceiling), gives figures for this split. It is a secondary source, so treat the split as indicative. ![A librarian at a long shelf of archive boxes pulling a single folder, retrieval from a document collection](/uploads/blog/stop-using-vector-dbs-for-agents-hierarchical-knowledge-graphs-vs-rag/gemini-supporting.jpg) ### What does GraphRAG cost to set up? Building the graph is the expensive part of GraphRAG, and the cost lands before anyone has asked a single question. The indexing steps above call a language model for entity extraction, description summaries and community reports, so cost grows with the size of the document pile. Microsoft has since addressed the cost. In a [November 2024 post on LazyGraphRAG](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/), the team says LazyGraphRAG indexing costs are "identical to vector RAG and 0.1% of the costs of full GraphRAG." It skips model-based extraction at indexing time, uses plain language processing to find concepts, and postpones model work until a question arrives. The same post reports that for global queries it matches GraphRAG Global Search answer quality at more than 700 times lower query cost. Those are Microsoft's own claims about its own method, so check what your installed version supports. Quality of the graph also depends on tuning. The docs warn that using GraphRAG out of the box "may not yield the best possible results" and recommend prompt tuning for your data. They also note that claim extraction is off by default because it generally needs tuning. ### What do AI agents need from retrieval? An agent is an AI system that takes several steps toward a goal, often looking things up along the way. Agents ask both kinds of question. "What is the refund window?" is a lookup. "Which suppliers are linked to the delayed orders?" asks the system to follow relationships across documents. The research does not test agents directly. The GraphRAG paper and blog evaluate question answering over document collections. Give the agent a plain vector search for lookups, which is cheap and fast when the answer sits in one or two passages, and a graph-based route for relationship and summary questions, which follows connections or summarises across documents. For agent control flow rather than retrieval, see [deterministic state graphs for AI agent workflows](/radar/deterministic-state-graphs-preventing-infinite-loops-ai-agent-workflows), and for the wider picture, [the rise of autonomous AI agents](/radar/rise-of-autonomous-ai-agents-multi-agent-workflows). ### What to watch next Independent head-to-head tests with real numbers would fill a gap, since the original evaluation relied on an AI grader and reported no scores. Cheaper variants such as LazyGraphRAG may land in the open-source package, and it remains to be seen whether they hold up on private data. Agent frameworks that pick between lookup and graph routes per question are the pattern to look for in how retrieval choices get wired in. ## Frequently asked questions ### What is GraphRAG in simple terms? GraphRAG is a retrieval method from Microsoft Research. A language model reads your documents, extracts the entities and relationships into a knowledge graph, groups related entities into communities and writes a summary for each. At question time the AI uses those summaries and connections instead of only fetching similar-looking text chunks, which helps with questions about a whole collection. ### What is the difference between GraphRAG and vector RAG? Vector RAG splits documents into chunks and retrieves the ones whose numeric fingerprints are closest to the question. GraphRAG first builds a graph of entities and relationships with community summaries, then searches those structures. Vector RAG suits questions answered in a passage or two, while GraphRAG targets questions that connect many documents or summarise an entire collection. ### Does GraphRAG replace a vector database? No. The official GraphRAG pipeline itself embeds entity descriptions, text units and community reports into a vector store, and the docs include a basic vector search mode for questions better suited to standard RAG. Teams usually treat the graph as an addition for relationship and summary questions, keeping vector search for simple lookups. ### Is GraphRAG expensive to run? Full GraphRAG calls a language model for entity extraction, description summaries and community reports during indexing, so cost rises with corpus size. Microsoft's LazyGraphRAG post says its variant has indexing costs identical to vector RAG and 0.1% of full GraphRAG. Those are the vendor's own claims. ## Source Based on Microsoft Research's GraphRAG paper. https://arxiv.org/abs/2404.16130 --- Published on SyntaxRadar (https://syntaxradar.com). Canonical URL: https://syntaxradar.com/radar/stop-using-vector-dbs-for-agents-hierarchical-knowledge-graphs-vs-rag. Last modified 2026-10-29T10:33:58.900Z.