Use a vector database when an agent’s answer sits in a few passages that read like its question. Use a knowledge graph when the answer depends on how people, documents, accounts and rules connect, and on facts that must match exactly. An agent that applies company rules usually needs both, with the graph deciding what applies and vector search helping it find where to start.
What is the difference between a knowledge graph and a vector database?
A vector database stores embeddings, which OpenAI’s guide defines this way: “An embedding is a vector (list) of floating point numbers. The distance between two vectors measures their relatedness” developers.openai.com. You split documents into chunks, embed each chunk, and at query time return the chunks whose vectors sit closest to the question’s vector. The original retrieval-augmented generation paper did this, splitting each Wikipedia article “into disjoint 100-word chunks, to make a total of 21M documents” and searching them as a dense vector index arxiv.org.
A knowledge graph stores things and the relationships between them. The 2020 survey Knowledge Graphs by Aidan Hogan and colleagues adopts an inclusive definition. In it, a knowledge graph is “a graph of data intended to accumulate and convey knowledge of the real world”. Its nodes represent entities of interest, and its edges represent relations between them arxiv.org. A query names a starting point and a pattern of edges to follow. In Neo4j’s Cypher, for example, a pattern “can be used to match patterns of a variable or an unknown length”, which covers jobs like finding every node reachable from an anchor or walking a hierarchy of uneven depth neo4j.com.
A vector index answers “what text is about this?”, and a graph answers “what is connected to this, and how?”
How do the two compare for an agent’s work?
We built this grid from the papers and docs cited on this page. It covers storage and querying, multi-hop and exact questions, provenance, freshness and build cost.
| Approach | What it stores and how it is queried | Multi-hop and exact constraints | Provenance and freshness | Cost of building it |
|---|---|---|---|---|
| Trail | A context graph parsed from policies, SOPs and records; a query is matched by similarity and by entity, then the graph is traversed | Rules sit on entities and named edges, so scope and precedence come from structure; low confidence is reported with what is missing | Every node and edge cites document, page and text; a changed source is re-parsed and the affected rules re-proposed for reviewtrailhq.com | Managed product: free plan with 100 rules, paid plans from $20k a year; public developer docs are still a placeholder |
| Vector database | Embeddings of text chunks; nearest-neighbour search by distance to the query vector | Strong when the answer is in one or two chunks; numbers and IDs are weak signals, and pgvector's approximate indexes filter after the scan | Each chunk can carry its source; a changed document means re-chunking and re-embedding it | Low: chunk, embed, index. pgvector runs inside an existing Postgresgithub.com/pgvector |
| Knowledge graph | Entities and typed relationships; path queries in a graph language such as Cypher | Strong: follows links across many hops and matches property values exactly | Provenance only if you model it; edits are per node or edge | High: schema, extraction, entity resolution and upkeep are your work |
| GraphRAG | An LLM-extracted entity graph plus community summaries of related entities | Built and evaluated for questions about a whole corpus, not for exact thresholds or IDs | Answers reference text units; an update command refreshes an existing index | README warns indexing can be expensive and says the project is largely in maintenance modegithub.com/microsoft |
| Vector index in a graph or SQL database | Embeddings stored on graph nodes (Neo4j) or in table columns (pgvector) next to structured data | Similarity finds entry points; joins or path queries apply exact conditions | As good as the schema you design | Medium: one store to run, two query styles to maintain |
Facts in this table were checked against the sources listed at the end of this page, between and .
Why do multi-hop questions favor a knowledge graph?
Because the answer is spread across records that do not read like the question. Microsoft Research’s GraphRAG paper says the conventional approach, “which we collectively call vector RAG, works well for queries that can be answered with information localized within a small set of records” arxiv.org. An agent’s question is not always localized.
Take an accounts payable agent asked whether invoice 4471 can be paid with one approval. The answer needs the invoice, the vendor on it, the framework agreement that vendor signed, the clause in that agreement that sets an approval limit, and the AP policy that clause overrides. Only the clause and the policy contain words like “approval”. The invoice and the vendor record are where the path starts, and nothing in their text points at the agreement. A graph stores that path as edges, so the agent can walk from the invoice to the policy that clause overrides in four hops and show each hop.
The same paper finds a second weakness at the other end of the scale: RAG fails on “global questions directed at an entire text corpus”, its example being “What are the main themes in the dataset?” arxiv.org. That is more an analyst’s question than an AP agent’s.
Why do exact constraints trip up vector search?
Embeddings measure relatedness, and two clauses that differ only in a number are closely related. A clause requiring a second approver above $10,000 and one allowing a single approver up to $25,000 are likely to land near each other, and near the question, while saying opposite things about an $18,400 invoice. The model then has to decide which governs, and retrieval gives it no hint.
One fix is a metadata filter, such as vendor or document type, to cut the candidates. The pgvector README describes the catch: “With approximate indexes, filtering is applied after the index is scanned. If a condition matches 10% of rows, with HNSW and the default hnsw.ef_search of 40, only 4 rows will match on average” github.com/pgvector. The extension offers iterative index scans to fetch more, and its default exact search “provides perfect recall”, so you choose where to sit between speed and completeness.
On FEVER, a fact-checking benchmark, the RAG authors found that plain BM25 keyword retrieval did better than their learned dense retriever, “perhaps since FEVER claims are heavily entity-centric and thus well-suited for word overlap-based retrieval” arxiv.org. Questions that turn on names and identifiers reward exact matching. A graph goes one step further and stores the entity as a node, so “Northwind Freight Ltd” in the contract and “NW Freight” in the ERP can resolve to one thing.
Which approach keeps sources and stays current?
Either store can cite sources if you build it to. The RAG paper’s abstract lists “providing provenance for their decisions and updating their world knowledge” as open problems for models that keep knowledge only in their weights arxiv.org, and an external store is how both approaches address them.
A vector store keeps provenance at the level of the chunk. If each chunk carries its document and page, the agent can cite where the text came from. It cannot cite why two chunks together produce an answer, because the link between them was made by the model at read time. Freshness is simple in principle: the paper showed “we can update RAG’s world knowledge by simply replacing its non-parametric memory”. In practice you re-embed changed documents, and chunks from an old policy version keep matching until someone deletes them.
A graph can keep provenance on every node and edge, including the links a model would otherwise infer, if the schema is designed for it. When a source changes, the affected nodes and edges change too, and a graph can keep the old version alongside the new one so last quarter’s decision stays explainable. The cost is that someone, or some pipeline, has to work out which nodes and edges a changed paragraph touches.
What does each approach cost to build?
A vector index is cheap to start. Chunk the documents, call an embedding model, load the vectors. The pgvector pitch is “Store your vectors with the rest of your data”, with ACID compliance, point-in-time recovery and JOINs from Postgres github.com/pgvector. A team already on Postgres adds no new system.
A knowledge graph costs more up front. Someone decides the entity types and edges, writes extraction, resolves duplicate names across systems and keeps it all current. LLM extraction speeds up the first pass and moves the cost into prompts and review. GraphRAG’s README warns that indexing “can be an expensive operation” and tells users to “start small” github.com/microsoft. Its paper notes that longer chunks need fewer LLM calls but “suffer from degraded recall of information that appears early in the chunk” arxiv.org. That cost is easier to justify when several agents reuse the same entities and rules.
Can you use a knowledge graph and a vector database together?
Yes. The common pattern uses similarity to find where to start and the graph to decide what applies, in one of three designs.
GraphRAG builds the graph from text. An LLM derives “an entity knowledge graph from the source documents” and then pregenerates “community summaries for all groups of closely related entities”; each summary drafts a partial answer and the partial answers are combined arxiv.org. The paper tested it on podcast transcripts of about 1 million tokens and news articles of about 1.7 million.
Neo4j puts a vector index inside the graph database. Its manual says vector search “is often used on its own for semantic matching, but it can also be combined with full-text search or other ranked result sources for hybrid search”. Since version 2026.01 the Cypher SEARCH clause queries vector indexes with filters on node properties neo4j.com. One caution from the same page: scores “are meaningful within the result set from a vector query”, so rank each source on its own instead of comparing raw scores across them.
pgvector takes the relational route. Embeddings live in a column, foreign keys carry the relationships, and a WHERE clause or a join applies exact conditions. The README points to Postgres full-text search for hybrid search, with Reciprocal Rank Fusion or a cross-encoder to merge the results github.com/pgvector. Multi-hop paths need a recursive WITH query, which the Postgres docs say is “typically used to deal with hierarchical or tree-structured data” postgresql.org, and is harder to write than a Cypher path.
None of the three, as shipped, records which of two matching rules wins, who approved a rule, or whether a rule is still in force; you model that yourself. For vendors that sell a graph with some of that built in, see our comparison of context graph tools for AI agents.
How Trail’s context graph uses both similarity and entities
Trail, the memory and context layer for enterprise AI agents, builds its graph around company rules. Its Context Graph page says sources are “Parsed into structure, not flattened into vectors”, and that “Every node and edge carries the document, page and text it was drawn from” trailhq.com. Matching a query still uses similarity. The feature walkthrough says a query “is matched two ways: by similarity and by entity”, after which “The graph is traversed to pull in nearby nodes”. Episodes and rules come back as candidates, and a reranker decides what goes back to the agent trailhq.com.
On the invoice question above, that means the vendor’s different names resolve to one node. How it works says “The same customer named four ways across four systems resolves to one node”, and an edge is “a stated claim with the sentence it was drawn from attached” trailhq.com. Rules reach agents only after the team approves them. When a source changes, Trail re-parses it and re-proposes what it affects for review. Where confidence is low it names what is missing, such as an unapproved exception or a stale source. An agent reads it over MCP, through a LangChain or LlamaIndex retriever, or by REST and GraphQL trailhq.com.
Trail is the wrong layer if you need a general graph database for your own data model, or a vector store for semantic search over support content. Public developer docs are not out yet (trailhq.com/docs says the page is on the roadmap) trailhq.com, and paid plans begin at $20k a year trailhq.com. Trail’s own blog also says a context graph needs no special database: “A graph database works, and so do relational tables with an embedding index for similarity search” trailhq.com.
How does one invoice question play out with each approach?
We worked the invoice example through each design on paper; no product was run. The facts: invoice 4471 from “NW Freight” for $18,400. AP policy section 5 says invoices above $10,000 need a second approver. Framework agreement FA-7 with Northwind Freight Ltd says invoices under it up to $25,000 need one approver, and a later memo says FA-7 overrides section 5 for that vendor.
| Approach | What reaches the agent | What the agent still has to work out | Likely result |
|---|---|---|---|
| Trail | Matched by similarity and by entity; traversal reaches FA-7 through the resolved vendor node; ranked, cited rules come back | Nothing, if the override was parsed and approved; if not, the answer says what is missing | One approver, citing FA-7 and the memo, or a stated gap |
| Vector database | Policy section 5 and probably the FA-7 clause, as two similar chunks | Which vendor FA-7 covers, whether NW Freight is Northwind, which rule wins | Possibly two approvers, if the model follows section 5 |
| Knowledge graph | Invoice to vendor to FA-7 to clause to the override edge | Nothing, if every entity and edge was modeled and kept current | One approver, with the path as evidence |
| Vector index in a graph or SQL database | The FA-7 clause via similarity, then the vendor link via a join or path | Precedence, unless the schema stores it | One approver if the override is a field; a guess otherwise |
Facts in this table were checked against the sources listed at the end of this page, between and .
The graph rows depend on someone having stored the override as data. If it lives only in a memo nobody parsed, every design falls back to the model reading two clauses and choosing.
Which should your agent use?
Start from the hardest question the agent must get right, then add structure only where that question needs it.
A decision rule for knowledge graph vs vector database
Step list- The answer sits in one or two passages that read like the questionUse a vector index. pgvector inside Postgres is enough for many teams; keep source and page on every chunk.
- The question names IDs, amounts, dates or vendorsAdd exact filters or keyword search beside vectors, and check how many rows survive an approximate index filter.
- The answer needs three or more linked recordsModel those entities and links as a graph, in a graph database or in relational tables, and use similarity only to find the entry point.
- The question is about themes across a whole document setThe GraphRAG paper found its community summaries beat a conventional RAG baseline on comprehensiveness and diversity for such questions. Budget for the LLM calls that indexing takes.
- Two rules can match and one must win, with an approver on recordStore precedence, approval and version as data on the graph, or use a governed rules layer such as Trail.
If the agent also remembers users and past sessions, that is a separate layer, covered in what agent memory is and our comparison of memory layers for AI agents.
How we compared knowledge graphs and vector databases
On 20 October 2026 we read the retrieval-augmented generation paper by Lewis et al., the GraphRAG paper by Edge et al. and the knowledge graph survey by Hogan et al. on arXiv. We also read the pgvector README, the Postgres docs on WITH queries and Neo4j’s Cypher manual pages on vector indexes and variable-length paths. The GraphRAG README and docs, OpenAI’s embeddings guide and Trail’s product pages were checked the same day. Every quote was fetched with curl and matched against the source text. The criteria were what each approach stores, how it is queried, multi-hop and exact-constraint handling, provenance, freshness and build cost. The invoice example is our own illustration; we did not build or benchmark any of these systems. Trail publishes this page.
Frequently asked questions
Knowledge graph vs vector database: which should AI agents use?
Use a vector database when the answer sits in a few passages that read like the question, such as support articles or meeting notes. Use a knowledge graph when the answer depends on how things connect, such as which agreement covers this vendor and which rule it overrides. Agents that apply company rules usually need both.
Can you use a knowledge graph and a vector database together?
Yes. A common pattern uses vector search to find the entry points and graph traversal to collect what is connected to them. Neo4j stores vector indexes on graph nodes, pgvector puts embeddings in Postgres tables next to relational data, and GraphRAG builds a graph from text and answers from community summaries.
Is GraphRAG a knowledge graph or a vector database?
GraphRAG is a pipeline that uses a language model to extract an entity knowledge graph from documents and then writes summaries of related groups of entities. It answers questions about a whole corpus from those summaries. Its README says the project is largely in maintenance mode.
Do I need a separate graph database to build a knowledge graph?
No. A graph database such as Neo4j makes path queries easy to write, but relational tables with foreign keys and an embedding column can hold the same entities and links. The modeling work, entity resolution and source tracking matter more than the storage engine.
Why does vector search miss exact constraints such as amounts and dates?
An embedding measures how related two pieces of text are, so a clause about invoices over $10,000 and one about invoices over $25,000 sit close together. Filters on metadata help, but with an approximate index pgvector applies the filter after the scan, which can return fewer rows than asked for.
How does an ontology relate to a knowledge graph?
An ontology defines the kinds of things and relations in a domain, and a knowledge graph holds the actual entities and links. Tom Gruber's 1993 paper defines an ontology as 'an explicit specification of a conceptualization'. The Hogan survey adds that simple statements become edges in the graph, while general ones such as 'all capitals are cities' call for an ontology or rules.
Sources checked
Show 19 sources
- Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (arXiv)
- Edge et al., From Local to Global: A Graph RAG Approach to Query-Focused Summarization (arXiv)
- Hogan et al., Knowledge Graphs (arXiv)
- Gruber, A Translation Approach to Portable Ontology Specifications (1993)
- OpenAI embeddings guide
- pgvector README
- Neo4j Cypher Manual: vector indexes
- Neo4j Cypher Manual: variable-length paths
- PostgreSQL docs: WITH queries
- Microsoft GraphRAG README
- GraphRAG docs
- GraphRAG CLI reference
- Trail Context Graph
- Trail Context Graph: how it works
- Trail feature walkthrough
- Trail homepage
- Trail pricing
- Trail docs placeholder
- Trail blog: how to create and use a context graph