Benzi Product comes out on top (9.5/10), followed by Benzi MCP (8.8), CodeGraph (8.2), and Bare CC (7.7).
Overview & comparative ranking
| Rank | Arm | Avg score | Strengths | Weaknesses |
|---|---|---|---|---|
| 1 | Benzi Product | 9.5 / 10 | Exceptional depth, includes concrete line numbers and internal state flags (abortIndex, no_grammar gotchas, OwnedTasks shut-down race), and clear architecture diagrams/tables. | Slightly higher output token overhead (though focused entirely on useful code details). |
| 2 | Benzi MCP | 8.8 / 10 | Very accurate, complete step-by-step traces, excellent handling of asynchronous event loops and memory model details (e.g. Tokio Header/Trailer state transitions). | Lacks some of the ultra-deep structural breakdowns and code-level nuances found in Benzi Product. |
| 3 | CodeGraph | 8.2 / 10 | Clean formatting, clear summary sections, strong direct code citations. Very solid everyday technical answers. | Slightly higher-level on complex runtime questions (e.g. missing some inner state-machine details in Tokio/Alamofire). |
| 4 | Bare CC | 7.7 / 10 | Accurate code references, concise and direct. | Can be overly terse, misses secondary runtime guarantees (memory management/shutdown details, edge-case invariants) compared to the top arms. |
Benzi Product comes out on top (9.5), followed by Benzi MCP (9.0), Bare CC (8.9), and CodeGraph (8.6).
Overall average and ranking
| Rank | Arm | Avg score | Notes |
|---|---|---|---|
| 1 | Benzi Product | 9.5 | Most complete and precise overall. Consistently includes exact call paths, edge cases, and useful summaries. |
| 2 | Benzi MCP | 9.0 | Very strong, accurate, and well-structured. Slightly less exhaustive than Benzi Product on several repos. |
| 3 | Bare CC | 8.9 | Concise, accurate, and usually complete. Very competitive; fewer deep edge-case details. |
| 4 | CodeGraph | 8.6 | Generally excellent, but the Gin answer contains a notable error: it says a middleware that never calls Next() stops the chain. In Gin, it does not necessarily stop; Abort() is what short-circuits. |
Key scoring observations
- Benzi Product wins on depth: it usually gives the full pipeline, exact functions, code snippets, and summaries. Its answers are the most complete.
- Benzi MCP is close behind and often cleaner. It gives correct details without excessive length.
- Bare CC is highly reliable and often very well organized. It does not always go as deep, but no major factual errors were found.
- CodeGraph is mostly very good, but the Gin
Next()/middleware-stop claim is a real correctness issue, which lowers its overall score.
Benzi Product comes out on top (avg 8.7). Bare CC and Benzi MCP tie at 8.0, and CodeGraph is last at 7.7.
Score by repo
| Repo | Bare CC | CodeGraph | Benzi MCP | Benzi Product |
|---|---|---|---|---|
| gin | 7 | 6 | 8 | 9 |
| alamofire | 8.5 | 7.5 | 7 | 8 |
| okhttp | 8 | 9 | 8.5 | 8.5 |
| excalidraw | 8.5 | 8 | 7 | 9 |
| tokio | 8 | 7 | 9 | 8.5 |
| django | 8 | 8.5 | 8.5 | 9 |
| Average | 8.0 | 7.7 | 8.0 | 8.7 |
Bare CC is Claude Code, Sonnet 5. CodeGraph is bare CC + CodeGraph MCP. Benzi MCP is bare CC + Benzi MCP. Benzi Product is Benzi's own standalone agentic loop (running on Claude Sonnet 5 as well, but no Claude Code). Benchmark methodology replicated from CodeGraph's own published benchmarks: github.com/colbymchenry/codegraph
Our read: we've already shown Benzi is considerably cheaper (2x) and faster (40%) than bare Claude Code on bug-fixing tasks (click here). Code intelligence cutting the cost of localizing and fixing a bug, while raising the cost of an open-ended architectural question, tracks with intuition — the same tools that shortcut a targeted fix invite more exploration on a broad "how does this work" question. Benzi's answers also generally cover more depth, or connect systems that bare Claude Code and Claude Code with CodeGraph as an MCP tool miss.