SEARCH FOR
AI AGENTS.
A new way for AI to search for data: one local Rust binary that indexes your code, docs, specs and logs, plus Corpus Hub packs, and returns cited answers. The binary is Apache-2.0. It answers with file:line citations and serves them through a CLI, an MCP server and the Elasticsearch API, so an agent reads a passage instead of a whole file.
THREE KINDS
OF USER.
xerj init writes the MCP config and the skill files for the agent directories it finds, and keeps a .bak of any file it edits.xerj autoindex <folder> sniffs each file by content. xerj autoindex map prints what landed. xerj autoindex remove --prefix <name> drops that folder after printing a plan.Not a fit, today: neural semantic search with no extra flag (the default embedder is lexical feature hashing; --embed-mode neural loads an all-MiniLM-L6-v2 model, about 90 MB, at startup), regex search over source (the benchmarks page marks it “not offered”), or a machine where an idle daemon must cost nothing (8% CPU and 8.3 GB on a 15-repo corpus, a published loss).
INSTALL, INDEX,
ASK.
The installer at xerj.org/get picks the release for your OS and CPU, checks its SHA-256, and puts xerj on ~/.local/bin (xerj.exe there under Git Bash, MSYS or Cygwin). PowerShell's installer (get.ps1) puts xerj.exe in %LOCALAPPDATA%\Programs\xerj instead. On Linux the default archive is the static musl build. xerj --version on that binary, run 2026-10-23, printed xerj v1.0.0-rc.98.
curl -fsSL https://xerj.org/get | sh
xerj --insecure --data-dir ~/xerj-data & # node on 127.0.0.1:9200
git clone --depth 1 https://github.com/psf/requests ref/requests
xerj autoindex ref/requests --prefix req
xerj search --prefix req -k 3 "rebuild auth on redirect strip authorization"
xerj def --prefix req should_strip_auth
--insecure turns off TLS and API-key auth. It is for one user on a dev machine. The default bind is 127.0.0.1. Without the flag, clients send the key the node writes to <data-dir>/admin.key.
No clone: ask a hub corpus
xerj "<question>" is the same command as xerj ask. It routes the question to a corpus in the signed catalogue, starts a local node when nothing answers on port 9200, pulls that corpus’s pack once, and names the corpus, the pin and the licence. This is the rc.98 binary on 2026-10-23, scratch home, feedback disabled. Stderr first, then the first hit. The rest of stdout is trimmed.
$ xerj "how does lz4 find a match" xerj hub: no node answers at http://localhost:9200; starting a local node on port 9300 (…) xerj hub: node ready on port 9300 (pid 3727) xerj hub: pulling 'lz4' (4.4 MB) — one-time download, later questions answer locally xerj hub: pulled lz4 0774d055 4623814 → 4 indices, 1702 definitions lz4 pin 2026-10-02 (21 days old) mixed: BSD-2 (lib) / GPL-2.0+ (rest) lz4/lib/lz4.c:939 LZ4_FORCE_INLINE int LZ4_compress_generic_validated( … approach-only licence (mixed: BSD-2 (lib) / GPL-2.0+ (rest)): read, do not copy
Same run also cited lz4/doc/lz4_Block_format.md:0. The node was stopped with xerj hub stop (stopped the hub node (pid 3727)). Catalogue entry for lz4, fetched the same day: pack_bytes 4623814, pin_sha 0774d055…, pin_date 2026-10-02.
ONE BINARY.
ONE CLI.
The rows below are the commands xerj --help printed on v1.0.0-rc.98, plus xerj hub --help. On that binary the top-level usage block lists xerj ask and does not list xerj hub. v1.0.0-rc.99 lists xerj hub next to xerj ask when the build has a hub signing key (#2282). xerj hub --help is the subcommand help.
| Command | What it does |
|---|---|
xerj --insecure --data-dir DIR | Starts the node. Elasticsearch-compatible API on 9200, native REST on 8080, gRPC on 8081. --port N moves the three to N, N+1 and N+2. |
xerj autoindex | <folder> discovers and indexes a folder or an s3:// / r2:// prefix. map prints the data map. remove --prefix <P> drops one autoindexed folder after a plan. |
xerj search · xerj def | Ranked passages with path:line, and go-to-definition. Lexical by default: they match the words you typed. |
xerj "<question>" · xerj ask | Answers from a hub corpus. --def SYMBOL, --how, --files, --offline, --json, --hosted on|off|auto. Exit 1 when nothing matches. |
xerj code · xerj corpus | corpus add|index|list pins and builds corpora you choose. code <corpus> "<query>" retrieves a definition and warns on the licence. |
xerj hub | sync, pull, feedback, status, stop, and config hosted on|off|auto. Packs are verified, then restored. They are not cloned or indexed on your machine. |
xerj mcp | On main it serves 23 tools, or 20 in a build with no hub key (mcp-tools.json). The hub three are xerj_hub_find, xerj_hub_def, xerj_hub_ask. It does not start a node. |
xerj init | Writes .mcp.json, and, when the directory exists, .cursor/mcp.json, .codex/config.toml, the skill and rule files, and an AGENTS.md section. |
xerj brain · xerj share · xerj gain · xerj feedback | A browsable second brain, a read-only link with a passcode and an expiry, usage counted from the audit log, and the field report in section 07. |
The 20 names from that tools/list: xerj_search, xerj_semantic_search, xerj_vector_search, xerj_hybrid_search, xerj_memory_store, xerj_memory_recall, xerj_brain_ego, xerj_brain_link, xerj_brain_unlink, xerj_brain_overview, xerj_map, xerj_index_budget, xerj_autoindex_remove, xerj_code_search, xerj_plan, xerj_field_report, xerj_corpus_feedback, xerj_hub_find, xerj_hub_def, xerj_hub_ask. The schema on main after the rc.98 tag adds xerj_def and lists 21 (mcp-tools.json). That tool is not in the rc.98 binary. The agent surface is also on For agents and in llms.txt.
PACKS, SIGNED
AND PINNED.
xerj ask reads catalogue.json (ed25519 signature in the sibling .sig). Fetched 2026-10-23: generated_at 2026-10-22T07:11:17Z, format_version 1, 497 corpora, every one with a pack_url. Eleven domains. 490 are adapt-with-attribution and 7 are approach-only. A binary with no hub public key downloads nothing (exit 5). A pack that fails verification exits 3./corpus/ links in the file, and the opening line says 514). Seventeen of those slugs are not in the signed catalogue: agent-session-trajectories, concourse, counsel-chat, exploit-pocs-2023, exploit-pocs-2024, exploit-pocs-2025, exploit-pocs-2026, exploitdb-src, fluent-bit, freebsd-handbook, ghsa-db, github-api-description, mtl-bioinformatics-ner, nuclei-templates, rust-vulns, xerj-blogposts, xerj-columnar. A registry corpus that is not in the catalogue is added with xerj corpus add and indexed with xerj corpus index.xerj hub feedback <slug> --usefulness 1-5 --found yes|partial|no and the MCP tool xerj_corpus_feedback send the review. Both are in the rc.98 binary (#1886 merged 2026-10-20). GET /api/corpus-feedback on 2026-10-23 returned prior.mean 3, prior.weight 5, and scores for 3 corpora (awesome-ai-system-prompts, brex-prompt-engineering, prompt-engineering-guide), each score 3.17, reviews 1, confidence “very low”. zstd returned reviews 0 and score null. The Corpus Hub v2 post describes the score as a Bayesian mean with that prior.REAL OUTPUT,
REAL NUMBERS.
The index-and-search block is the capture in benchmarks/agent-intents-2026-10-19, run 2026-10-19 on xerj v1.0.0-rc.97 and ripgrep 14.1.1, psf/requests at 611c6162. It was not re-run for this page. The commands are the same ones rc.98’s help lists.
$ xerj autoindex ./requests --prefix req
done in 2.9s — 1 dataset, 1571 records live, 0 duplicate aliases, 0 junk records, 1 junk/skipped file (543.9 docs/sec)
xerj-done ok=true exit=3 reason=completed-with-junk wall=3.0s files=95 records=1571
$ xerj search --prefix req -k 3 --full 400 "rebuild auth on redirect strip authorization"
xerj search: these hits are lexical; this command does not consult the embedding.
─── src/requests/sessions.py:309 (score 22.97)
def rebuild_auth(
…
3 passage(s) from 'req-*'. Cite ax_path:line for anything you rely on.
| Same task, captured 2026-10-19 | Output an agent reads | Note |
|---|---|---|
rg -n -i redirect | 282 lines · 25,150 bytes | 17 files. File: 01-rg-redirect.txt |
xerj search … -k 3 --full 400 | 1,303 bytes | First hit rebuild_auth at sessions.py:309, score 22.97 |
xerj def should_strip_auth | 949 bytes | First hit sessions.py:154. The file also lists other symbols in that module |
| Wall time, 5 runs, two sessions | search 8 to 12 ms, then 19 to 23 ms | rg 5 to 7 ms in both sessions. README timing table |
The other phrasing tried that day, “strip the Authorization header when a redirect changes host”, ranked tests/test_requests.py (score 32.70), rebuild_method at sessions.py:370 (30.39), and rewind_body at utils.py:1139 (25.31). rebuild_auth was not in those three hits. Search is lexical by default, so the words matter. File: 01-xerj-search-first-phrasing.txt.
Published benchmarks
| Measurement | XERJ | Comparison | Source |
|---|---|---|---|
| Output tokens, 8 tasks × 4 languages, 16 runs per arm | 9,982 | 26,477 grep-driven · 260,916 memory only | benchmarks §04 |
| Cost of that matrix | $1.58 | $3.27 grep-driven · $11.18 memory only | benchmarks §04 |
| Unfamiliar-API tasks, hidden tests | 21 / 21 | 1 / 21 from model memory · $21.90 vs $3.38 | reference-coding case study |
| 88-cell board vs Elasticsearch 8.13.4 | 55 wins | 26 ties · 4 losses · 3 n/a | ES scorecard · demo/playbooks/SCORECARD.md |
| On-disk size, same docs | 176.2 MB | ES 283.0 MB | SCORECARD.md storage row |
| Identifier lookup, 15-repo corpus, p50 of 21 | 11.7 ms (one index) | Zoekt 15.3 ms · ripgrep cold 260 ms | benchmarks §01, rc.70 binary, 2026-08-29 |
Hybrid nDCG@10, SciFact / NFCorpus, --embed-mode neural | 0.6993 / 0.3448 | BM25 0.6572 / 0.3016 | benchmarks §03. FiQA hybrid: not run |
| Read p99 under a 40k docs/s writer | 10.3–13.6 ms | ES 3.5–6.8 ms | 4 LOSE cells on the scorecard (10.27–13.57 ms vs 3.45–6.76 ms) |
| Idle daemon, 15-repo corpus | 8% CPU · 8.3 GB | Zoekt 0.03% · 119 MB | benchmarks §01, #874 |
Write-ups, losses included: Does XERJ beat JEV?, a BM25 vote against Jev, cutting the index 28%, and the rc.80 gates.
WHAT ELSE
YOU COULD USE.
| Tool | Where it is ahead | Where XERJ is ahead |
|---|---|---|
| grep / ripgrep | No index, regex, and lower per-call time on a small tree (5 to 7 ms vs 8 to 23 ms on the requests capture) | Ranked passages and definitions with file:line. On that capture, 1,303 bytes against 25,150 |
| Zoekt / Sourcegraph | Idle footprint, and multi-repo fan-out on the 15-repo board | One binary that also indexes data files and serves MCP, memory and the hub. Definition ranking without ctags |
| Elasticsearch / OpenSearch | Read p99 under a 40k docs/s writer, and the plugin ecosystem | No JVM. The default Linux install is a static musl binary. Agent commands, MCP and the hub are in that binary, on the same wire API. 55 wins on the 88-cell board |
| A vector database | A neural index with a model you choose, at a scale you operate | BM25, kNN and hybrid in one process. The default embedder needs no API key. Neural mode is a server flag |
| GitHub code search | Every public repository, no download | An answer from a pinned, signed pack, offline after the first pull, with the licence on the hit |
Longer comparisons: XERJ vs ripgrep for code agents, vs Elasticsearch on a laptop, vs a vector database, and all comparisons.
ONE SHORT
REPORT.
xerj feedback drafts a field report and fills in only what it can observe. xerj feedback --submit opens a GitHub pull request that adds one file under user-feedback/16-agent-field-reports/, and posts to xerj.org/api/field-report only when GitHub is unavailable. --disable-feedback or XERJ_DISABLE_FEEDBACK=true turns the invitation off. The same switch turns off corpus-feedback reviews.
On main at this rewrite, that directory holds 52 report files besides the README, dated 2026-08-18 through 2026-10-23.
QUESTIONS
PEOPLE ASK.
xerj hub config hosted defaults to auto: off inside a git repository, on elsewhere. --hosted off or --offline keeps the question local. The optional Jev rerank stage is the search-time feature that sends document text off the node, and it stays off until an operator sets a key (llms.txt).xerj --help calls the default embedder lexical feature-hash, offline, not neural. --embed-mode neural loads all-MiniLM-L6-v2. proxy points at an OpenAI-compatible embeddings endpoint.xerj init writes Claude Code, Cursor and Codex config when those directories are present. Other clients use the mcpServers block xerj mcp --help describes.xerj ask cover any GitHub repo?xerj autoindex. Asking an arbitrary repo URL with no index step is #1800, still open.