Four things a search engine for AI agents has to prove, each measured, each linked to the
harness or the case study that produced it: engine performance,
code retrieval, token savings, and the design that ties them
together — the zero-token architecture. The red cells are published too, root-caused with
file:line, because a benchmark page that hides its losses is marketing.
Two performance stories matter for an engine that lives next to your agents: how fast a query comes back, and what the process costs when nobody is asking. Both are measured; neither is extrapolated.
| Measurement | XERJ | Comparison | Verdict | Source |
|---|---|---|---|---|
| 88-cell query board vs Elasticsearch 8.x — reads, aggs, pipelines, mixed, kNN, storage | 55 wins | 26 ties · 4 losses · 3 n/a | WIN | scorecard — one win is honestly a draw; the page says which |
Server round-trip, size:0 over 300k docs, keep-alive client |
0.126 ms | ES 0.283 ms | 2.2× | scorecard · read transport |
| Bulk ingest, identical corpus and box | 1.72× higher | ES baseline | WIN | scorecard |
| On-disk size, same docs | 176.2 MB | ES 283.0 MB | 1.61× smaller | scorecard · storage caveats apply |
| kNN k=10, HNSW-served with exact rescoring | 1.76 ms | ES 2.08 ms | TIE (1.18×, inside noise) | scorecard — recall is measured, never assumed |
| Identifier lookup, one index, 15-repo code corpus (p50 of 21) | 11.7 ms | Zoekt (Sourcegraph) 15.3 ms · ripgrep cold 260 ms | WIN per-index | 2026-08-29 session, same box, released rc.70 binary |
| Mixed read-under-write p99, 40k docs/s iso-load writer | 10.3–13.6 ms | ES 3.5–6.8 ms | 4 LOSSES | scorecard · root-caused to a lock, fix targeted |
| Dimension | XERJ rc.70/71 | Zoekt (Sourcegraph) | ripgrep | What it means for you |
|---|---|---|---|---|
| Identifier search, p50 | 11.7 ms (one index) | 15.3 ms | 260 ms cold | An agent that retrieves 50 times in a session waits ~0.6 s total, not 13 s — retrieval stops being the slow step in the loop. |
| — across all 41 datasets today | 701 ms | 15.3 ms | 260 ms | Our multi-index fan-out is serial today — measured, filed, fix scoped (#875: expected ~20–30 ms). |
| Exact phrase, p50 | 829 ms (fan-out) | 37.9 ms | 200 ms | Same #875 fan-out defect; the per-index engine is competitive once the query reaches it. |
| Regex over source | not offered | 870 ms | 180 ms | Nobody owns this today — Zoekt's regex loses to cold grep at this corpus size. A trigram side-car is on our roadmap precisely because the category is open. |
| Index 15 repos | 676 s + merge tail | 302 s | 0 s (no index) | XERJ's indexer does strictly more work — AST symbols in 13 languages and graph edges, not just trigrams — but the merge tail is a defect, not a feature (#876). |
| Idle daemon, this corpus | 8% CPU · 8.3 GB | 0.03% · 119 MB | none | Today's honest loss. Root-caused to three mechanisms with a published budget — idle under 0.5% of one core at any index count (#874). |
| Symbol / definition ranking | built in, 13 languages | needs universal-ctags installed | none | One binary gives your agent definition-first ranking with no sidecar toolchain to install or keep in sync. |
| Beyond code search | ES wire · aggregations · kNN vectors · agent memory | code search only | grep only | The same daemon that answers your agent's code lookups holds its logs, vectors and memory — one thing to run instead of three. |
Method: released XERJ binaries, Zoekt built from source at HEAD, ripgrep 14.1.1; identical
15-repo corpus (memcached, valkey, tantivy, regex, zstd, CRoaring, …), p50 of 21 requests per cell, CPU and RSS
from /proc. Zoekt indexed serially per its defaults; XERJ's indexer also extracts AST
symbols and graph edges, which Zoekt does not attempt — stated so the wall-clock rows are read fairly. Result
quality (ranking, symbol precision) is not scored here.
file:line
(#871–#876).
That is the deal this page offers: competitors are named, the red cells get the same precision as the green
ones, and each red cell links to its fix.The retrieval claim is not "search is nice" — it is that retrieval makes the same model
correct on APIs it has never memorised. Measured across 13 purpose-built libraries in 5 languages (unfamiliar
by construction, so the model cannot bluff), with hidden-test verdicts and real token accounting from
claude -p.
xerj autoindex parses source
through tree-sitter grammars in 13 languages and emits every symbol with its kind and line, a searchable
defs field, and the full body — so an identifier query ranks the definition
first. No ctags, no sidecar toolchain: one binary. Extraction throughput is measured and published
(~1,500 files/s single-thread on a 6,113-file Lucene checkout), and an unchanged re-index skips the parse
for byte-identical files (~100× on the edit-and-rerun path, shipped in rc.71).file:line: a few hundred tokens
carrying the one thing a compiler can never leak, the runtime contract.Method, per-library table, and every per-run record: the reference-coding case study · docs/case-studies/reference-coding
Output tokens are the expensive kind — priced roughly 5× above input on Claude models — and
retry loops on unknown APIs burn exactly those. Three arms of the same agent on the same tasks: memory only,
grep-driven, retrieval-injected. Every figure from claude -p --output-format json.
The per-language medians run 6.7× (JavaScript) to 278× (Java) fewer output tokens than answering from memory. An independent 12-question dev-QA measurement on the engine's own reference corpora lands at 1.65× fewer output tokens and 1.47× cheaper — smaller, because it includes questions where the model already knew the answer; both numbers are published.
multi_match over defs/body/title, ~10 ms server-side.file:line — hundreds of tokens, not hundreds of thousands.The cheapest token is the one your model never generates. ZTA is the design rule that produced every number above: spend compute once, at index time, so agents stop spending inference tokens — the metered, per-request, 5×-priced resource — re-deriving what the index already knows. It is an architecture target with measured proxies, not a certification; here are its four principles and the number that keeps each one honest.
Parse the AST once — 13 languages, every symbol with kind and line, a ranked
defs field — instead of letting every future question re-derive structure with
tokens. Index-time compute is bought once; token-time compute is bought on every question, forever.
The unit of answer is the definition with its contract and file:line —
never "here are eleven files, good luck." Small answers keep the agent's context small, which compounds:
every later turn re-pays for everything already in the window.
Setup instructions live at llms.txt in machine order: install, start, index, query, retrieval discipline. One pasted sentence turns it on — tested verbatim, transcript published. Zero tokens spent negotiating with documentation written for humans.
An always-on corpus next to your agents must not tax the machine they work on. Zoekt sets the bar at 0.03% — the head-to-head above shows we are not there yet, and instead of hiding that, the remaining idle mechanisms are located, filed, and budgeted in public (#874). The rule the codebase now enforces: no per-index periodic work, ever.
Why this is the sales pitch and not a slogan: infrastructure used to compete on latency; agent infrastructure competes on your inference bill. A retrieval that answers in 10 ms and a few hundred tokens replaces a 15,000-token retry loop every time it fires — the $21.90 → $3.38 delta above is that substitution, measured 21 times over. ZTA is the commitment that every future XERJ feature is judged by the same question: how many tokens does it stop your model from spending?