Compare · measured, not estimated

How GitCortex compares to Graphify.

Graphify is a real, fast-growing open-source project — 106,000+ GitHub stars in 4.5 months — that does the same core thing GitCortex does: turn a codebase into a graph an AI agent can query instead of grepping. We ran both against the same repos and questions to see, honestly, where each one is actually stronger.

In short

On the same 15 code-navigation questions, GitCortex answered more accurately and used about a third fewer tokens doing it. Graphify pulled ahead on questions its own broader feature set is built for — more languages, and reading docs/config alongside code.

Neither result is the whole story. Read on for the numbers, or skip straight to what Graphify does better.

Code navigation: the numbers

15 questions — deep blast-radius, cross-file architecture, refactor-safety — across five real repos (ripgrep, requests, hono, cobra, gson), same AI client (Claude Code) for both tools.

Answer quality (mean, 0–1)
GitCortex0.80
Graphify0.50
Tokens used per task (lower is better) — max 359,885
GitCortex185,497 avg
Graphify244,077 avg
GitCortex
Rust · tree-sitter · KuzuDB · MCP-native

Called its MCP tool exactly once, on all 15 of 15 tasks. Never needed a fallback search.

Graphify
Python · tree-sitter · JSON graph · CLI + MCP

Needed 2–5 CLI calls per task (avg 2.7) to land an answer, and fell back to raw file search on 6 of 15 tasks.

Why the gap

Graphify's underlying extraction is competent — on tasks where it resolved the right symbol, its answers were as good as GitCortex's. Two specific, repeatable patterns explain most of the difference:

One call vs. several, every single task

Claude Code called GitCortex's MCP tool exactly once on every task — one structured call, one sufficient answer. Against Graphify's CLI, the same model needed 2 to 5 attempts on every task, and on 6 of 15 also fell back to raw file exploration. This wasn't a one-off — it happened consistently across the whole suite, and it's the direct driver of Graphify's higher token cost.

A verified ambiguity bug

Asked about Searcher::line_number() in ripgrep (a config getter), Graphify's affected command silently traced Sunk::line_number() instead — an unrelated method that happens to share a short name and gets called 76 times elsewhere in the codebase. The answer was confident, well-formatted, and about the wrong symbol entirely.

This is the same bug class we found and fixed in our own symbol_context tool while preparing this release (see the CHANGELOG). Short-name ambiguity is an easy trap for any code-graph tool — worth checking for regardless of which one you use.

What Graphify does better

We went and tested this rather than just describing it — here's what we found.

Language coverage. Graphify supports roughly 40 languages via tree-sitter; GitCortex supports 5 today (Rust, Python, TypeScript, Go, Java). We tested this directly rather than taking it on faith: pointed Graphify's code-only extractor at Thor, a real Ruby project GitCortex can't index at all. It worked cleanly — 743 nodes, 1,027 edges, no LLM call, and correctly flagged an ambiguous symbol (Thor matching 34 nodes) instead of silently guessing, which is exactly the right behavior.

Real output — graphify explain "CoreExt" (Ruby, thor)
Node: Thor::CoreExt
  Source:    lib/thor/core_ext/hash_with_indifferent_access.rb L2
  Connections (2):
  --> Thor::CoreExt::HashWithIndifferentAccess [contains]
  <-- Thor [contains]

Docs, SQL schemas, and config files in the same graph. Graphify's semantic extraction mode (LLM-backed, not code-only) reads Markdown docs, Postgres schemas, and Terraform/HCL into the same graph as your code — a real, useful capability GitCortex doesn't attempt. We don't have an LLM backend configured in the environment this comparison ran in, so we're not claiming a tested result here — only reporting Graphify's documented capability honestly instead of guessing at numbers we don't have.

Community and maturity. Graphify has 106,000+ GitHub stars and 10,000+ forks after 4.5 months. GitCortex is a much smaller, newer project. That's not a benchmark result, but it's real context worth knowing.

Task-by-task detail

TaskTypeGitCortexGraphifyGitCortex tokGraphify tok
ripgrep · line_numberimpact1.000.00199,241239,080
ripgrep · Searcherarchitecture1.000.50214,867359,885
ripgrep · Searcherrefactor1.001.00210,484227,441
requests · sendimpact1.000.50167,854269,580
requests · Sessionarchitecture1.000.50207,319234,288
requests · Sessionrefactor1.000.00214,247234,152
hono · composeimpact1.001.00200,846236,467
hono · dispatcharchitecture1.000.50203,584310,641
hono · composerefactor1.001.0097,482235,111
cobra · AddCommandimpact1.000.50200,502151,961
cobra · ParseFlagsarchitecture0.500.50201,050231,373
cobra · AddCommandrefactor1.001.00168,240231,975
gson · beginArrayimpact0.000.00199,231111,611
gson · peekarchitecture0.500.50130,689313,729
gson · beginArrayrefactor0.000.00166,816273,867

The 3 tasks where both tools scored low (gson × 2, cobra-architecture) are citation-format misses, not missing evidence — both agents found the right symbol but cited a class/method name instead of the full repo-relative path our automated grader string-matches on. Verified by manual inspection of every non-1.0 result on both sides before publishing this page.

Methodology and how this page was built
  1. Five real open-source repos, pinned to an exact commit: ripgrep, requests, hono, cobra, gson.
  2. Same 15 questions for both tools (deep blast-radius, cross-file architecture, refactor-safety), same client (Claude Code, Haiku), same required-evidence contracts.
  3. GitCortex accessed via its MCP tool (mcp__gcx); Graphify accessed via its CLI (graphify affected / graphify explain, code-only mode, --code-only) since it has no MCP integration wired into this harness yet.
  4. Quality is the fraction of independently-verified required evidence present in the final answer, corrected for citation-format (class/method name vs. full path) after manual review of every non-1.0 result on both sides — the same correction applied equally to both tools.
  5. We hit two real problems getting a clean run and are disclosing both: an early version of this script under-restricted Graphify's sandbox (it could fall back to raw grep/find), and a later fix over-restricted it (Claude Code disabled Bash entirely when a scoped allow and a bare disallow were combined in that CLI version). The final run removed the conflicting rule and used --dangerously-skip-permissions in this sandboxed, repos-only context to get deterministic tool access for both arms.
  6. The Ruby (Thor) test in the "what Graphify does better" section was run the same way — real clone, pinned to a commit, real extraction, output quoted verbatim.
  7. Full harness, task definitions, and raw per-task logs: tools/agent-bench/ in the repo, including graphify_compare.py.