Answers

Short, direct answers on how AI agents search local files, code, logs and PDFs with XERJ. Each one has the commands to run and links to its evidence.

Agent access and memory

• Can ChatGPT search a folder on my laptop, or do I need something else? — xerj mcp is a stdio MCP server that proxies 23 tools to a running XERJ node, so ChatGPT, Claude or Cursor search an indexed local folder instead of opening files.
• I need my agent to find a passage and cite the file, not dump the whole tree. How? — Local code search MCP for Claude Code with file:line citations: index with xerj autoindex, call xerj_search. One captured call cited 2 symbols without opening files.
• Search tool for a Codex agent, on the files you have — xerj init writes the MCP entry for Claude Code, Cursor and Codex when those folders exist. On main, xerj mcp serves 23 tools, or 20 with no hub key.
• Should an agent use API search or vector search? — One question through 4 XERJ MCP tools returned 3 different result sets. Hybrid RRF returned 6 documents and merged both lists on the default lexical embedder.

Agent memory: concepts

• Agentic memory vs RAG: what is the difference? — RAG retrieves evidence for generation. Agentic memory stores information across tasks, often retrieved with RAG. Compare the roles and use both in one local node.
• What is agentic memory? — Agentic memory is memory an AI agent writes and reads itself through tools, so facts survive past one chat. What it is, the common types, and how to run it locally.
• What is an MCP memory server? — An MCP memory server gives an AI assistant tools to save and recall facts across chats over the Model Context Protocol. How it works and how XERJ serves one locally.

Agent memory: link retrieval

• Should agent memory use a knowledge graph? — XERJ builds a bounded link index over indexed documents, not a graph database. Our capture found 7 edges, answered 1 and 2 hops, and refused 3.

Agent memory: model dependency

• How do I stop an agent memory store from filling with near-duplicate notes? — Send dedup true on the store call and XERJ returns the existing id instead of a second document. The default lexical embedder needs no model; neural or proxy embedding is opt-in.

Agent memory: provenance

• How can agent-memory links retain evidence? — Every detected link in XERJ carries the exact line that taught it. All 7 edges in our capture had an evidence quote, and unlink wrote a tombstone.

Agent memory: security

• How do I give two agents private memory on one laptop? — XERJ keeps agent memory on local disk and made 0 outbound connections in our capture. Roles are stored but not enforced, so scoped API keys do the work.

Agent memory: setup

• How do people store user preferences so an agent can recall them next week? — XERJ keeps agent memory in a reserved index inside the data directory. We stored two memories, restarted the node, recalled both, then forgot one by id.
• Search Claude Code past sessions on disk — Index Claude Code session JSONL and search the message field. On a 2026-10-20 fixture, zstd returned 2 hits in message_content; xerj search returned none.

Agent memory: storage

• How do I set the MCP memory storage path? — XERJ has no separate memory path. Agent memory lives in the node data directory as .xerj-memory-{namespace}, and the MCP server stores nothing itself.

Agent memory: topology

• What's a simple way to give an agent long-term memory on my laptop without Qdrant? — Our capture ran agent memory on one XERJ process with three loopback sockets, no second data service, and a peak resident size of 100,928 kB.

Answers for coding agents

• Ask questions about a GitHub repo without cloning it — Run xerj ask and get answers about a GitHub repo with file:line, the pinned commit and the licence, from a signed pre-indexed pack. Nothing to clone.
• Batch lookup CVE IDs and GHSA IDs offline, without GitHub API rate limits — Load a signed advisory pack and look up CVE, GHSA and RUSTSEC IDs offline with source citations. The recorded six-ID run answered four and abstained on two.
• Full-text search over JSONL files from the CLI: index a folder and query it with BM25 — xerj autoindex reads JSONL and CSV with no schema, then answers BM25 per field from the CLI or MCP. 2,902 rows indexed in 12.4 s; queries took 1 ms.
• Local code search MCP for Claude Code that uses fewer tokens than grep — xerj indexes a repo once and returns ranked passages with file:line via CLI and MCP. On psf/requests one answer was 1,303 bytes; rg printed 25,150.
• Retrieval that says not found, and cites file and line when it does answer — xerj answers with file:line and exits 1 with no match when a symbol or corpus is missing. Free-text questions still return passages; confidence is #1786.

Batch lookup

• Look up 1,000 IDs against a local index — Put the IDs in one terms query. On 2026-10-20, 1,000 IDs against 532 issues returned took 0: 532 found, 468 absent.

Capabilities and limits

• Does XERJ support S3 storage, alerting or custom plugins yet? — Since v1.0.0-rc.77 the index CAN live in an S3-compatible bucket; since rc.78 record packs are real; since rc.80 _watcher evaluates exactly one watch shape and refuses every other with 501. Plugins stay built-in.

Claim checks

• Verify claims in a report against the saved sources — Index the saved sources and run one match_phrase per claim. On 2026-10-20, 3 of 5 phrases were in the docs index and 2 were not.

Code security

• How do I audit a PHP codebase with an index instead of grepping 1,400 files? — The published WordPress core audit indexed the tree as queryable facts, then read only the survivors. Core came back hardened, and that negative is the result.

Coding agents: reference retrieval

• How do I set up reference coding so the agent retrieves a mechanism before writing code? — Clone the library you are about to write against, index it with xerj autoindex, and retrieve the passage first. The published study, with the tracks where it loses.
• xerj corpus index --fresh failed. How do I rebuild a corpus index? — xerj corpus index --fresh builds a replacement index beside the old one, verifies it has records, switches the state file, and only then retires the old index. A failed build costs nothing.

Decisions as retrieval

• Can I route tickets or filter spam from labelled history, without an LLM call? — Classify text by voting over its nearest labelled examples, with no LLM or API call. Measured: 0.933 accuracy on 77-way intent routing, 0.983 on SMS spam.

Embeddings: cost

• How do I reduce embedding API cost? — XERJ embeds documents inside the node by default, so a captured run saw 0 non-loopback peers. Proxy mode sent exactly 2 requests for 100 documents.

Embeddings: local execution

• I want local search that works offline, no cloud embeddings. What are my options? — XERJ embeds locally in two ways. The default lexical embedder needs no model at all, and --embed-mode neural loads a cached MiniLM model on the CPU.

Embeddings: selection

• When should I turn on neural embeddings, and when is the lexical default enough? — Embeddings only helped when we opted into the neural embedder. On XERJ's lexical default, kNN promoted coffee storage above the synonyms for the query car.

Files and formats

• Count mentions of a word over time in a posts file — Index a JSONL posts file and send one date histogram on the date field. On 2026-10-20, autoindex in issue titles returned 88 hits and took 123 ms.
• How do I find every place a config key is set across YAML and XML in a repo? — XERJ autoindex indexes YAML and XML config files as full-text documents, maps 9 fields including body and title, and answers one bool query across both.
• How do I full-text search a SQLite database I just copied onto disk? — xerj autoindex reads a SQLite file read-only and turns each table into an index. A captured WAL-mode run indexed 400 of 450 committed rows and changed no bytes.
• How do I index a CSV with many columns? — xerj autoindex infers a mapping for a wide CSV with no schema work. A captured 120-column file produced 128 mapped fields and an exact terms aggregation.
• How do I index Markdown into the Elasticsearch API? — XERJ autoindex indexes Markdown as the txt-prose family and answers the Elasticsearch API, but it maps no headings array and keeps the hash in the title.
• How do I run syntax-aware code search? — XERJ indexes source files with a defs field and a symbols array, so one query finds declarations with kind, name and line while another finds raw text.
• How do I search a .csv.gz or .json.gz without unzipping it first? — Gzip is a wrapper, not a family. autoindex unwraps it and sniffs the inner CSV or JSON, so a compressed export becomes typed fields rather than one opaque blob.
• How do I search a Confluence export? — Point xerj autoindex at an unzipped Confluence HTML space export. A captured run kept page titles, nested page bodies, and the attachment tree.
• How do I search a Slack JSON export? — Point xerj autoindex at an unzipped Slack export. A captured run found a threaded reply, kept ts, thread_ts and user, and round-tripped Japanese text.
• How do I search BVH motion-capture files? — xerj autoindex reads .bvh files and writes frames, frame_time_s, duration_s, joint_count and a joints array you can query. Run one command on the folder.
• How do I search gzipped logs without decompressing the whole file — including when JSONL sits next to them? — XERJ decompresses a .gz log once at index time, so queries never read the archive again. The plain and gzip copies of one log returned the same counts.
• How do I search Kubernetes pod logs? — Collect pod logs to disk, then index the folder once. A captured run answered 1 phrase query across 3 pod log files and returned 18 hits, 6 per pod.
• How do I search through a folder of contracts in .docx? — Index a folder of Word .docx files with one xerj autoindex command, then search their paragraphs. A captured run also shows the decompression guard.
• How do I search Unity YAML scenes and the C# that references them as one project? — xerj autoindex detects the unity and unity-meta families, splits a scene into one index per Unity class, and joins an asset to its .meta record by path.
• How do I stop my agent from reading the whole repo into context? — Index the tree once with xerj autoindex, then let the agent query the defs and symbols fields instead of opening files. A dry run first prints every ignore rule.
• How should an agent figure out what's in a messy data folder before searching? — Point xerj autoindex at the folder, then read xerj autoindex map: every dataset it inferred, the fields and types, the time range, and every file it refused.
• I downloaded a bunch of papers as PDFs. How do I search across them? — One xerj autoindex command indexes the folder, then a match_phrase query reads every PDF. A captured run hit all 3 born-digital files and refused a scanned one.
• I have a 2GB SQL dump. How do I find rows mentioning a customer without loading it into Postgres? — xerj autoindex parses a MySQL-style .sql file directly, with no database server, and makes every INSERT value searchable. A captured run indexed 321 documents.
• I have a directory of CSV exports. How do I query them without opening each one in Excel? — Point xerj autoindex at a folder of CSV files once. A captured run split 3 CSVs with 2 headers into 2 indices and answered one query across both.
• I have a folder of PDFs, Word docs, and markdown. How do I search all of them at once? — Index a folder of PDFs, Word docs and markdown with one xerj autoindex command, then search the contents. A captured 6-file run indexed 51 documents into 2 indices.
• I have a JSON export of Jira issues (or Zendesk tickets). How do I search it like a help-center search? — Index the exported ticket JSON on disk. A captured run returned 36 description hits with per-ticket locators and an exact status count of 30 per status.
• I indexed a folder of email and PDFs. How do I actually read what it found? — xerj brain indexes the folder and the console opens any record: an email with its attachments, a PDF page, and the files it links to. Extracted text only.
• I need to find a function I wrote in a notebook months ago. How do I search all .ipynb files and the .py next to them? — Index a folder of Jupyter notebooks with xerj autoindex. Each heading section becomes one document with its heading path, starting cell and text output.
• I saved a help-center as HTML. How do I search it like the real help-center search? — XERJ autoindex indexes a static HTML export from local disk, extracts title and a headings array per page, and fetched 0 non-loopback peers in our capture.
• Is it efficient to index PDF files? — PDF extraction is the slowest family XERJ parses. XERJ runs it in an isolated worker sized from free memory, caps it 5 ways, and refuses image-only files.
• Is there a way to full-text search my browser history file itself? — Copy the Chrome History SQLite file and index it. A captured run made titles and URLs queryable, left the source unchanged, and missed 40 WAL rows.
• My agent needs to look up endpoints in a big OpenAPI spec and the SDK markdown. What's the right way? — Index the OpenAPI file with xerj autoindex. In a captured run a route name, a description and an operationId each returned 1 hit per serialization.
• What's the best way to search markdown notes and PDFs in the same vault? — A captured XERJ run indexed a vault of markdown notes and returned text from inside a PDF attachment and a DOCX attachment, plus wikilink edges between the notes.
• What's the easiest way to search JSON logs plus some old gzip text logs in the same folder? — xerj autoindex detects JSON arrays, JSONL logs and gzipped logs in one folder, infers the date field and answers exact filters. One run returned 386 ERROR hits.
• Why does my Notion export duplicate titles? — A captured run indexed a Notion Markdown export twice and got the same 8 documents with byte-identical ids. The duplicates come from the export layout.

Folder indexing

• Answer questions about a docs folder, with a file to cite — Index the docs folder with xerj autoindex. A phrase query returns the file to cite. xerj search on the same question ranked code lines instead.
• I exported my Gmail with Google Takeout. How do I search that mbox file locally? — Unzip the export, run xerj autoindex on the folder, and search messages, threads and attached PDF pages over HTTP. The mailbox is streamed, and with the default embedder no message text leaves your machine.
• I want to point something at a folder and then ask questions about what's in it. What should I use? — Index the folder with one command, read the map of what XERJ found, then let an agent query it over MCP instead of opening the files one by one.

GitHub issues

• Find duplicate GitHub issues before you file — Export issues to JSONL, index them with xerj autoindex, and search the draft title. On 2026-10-20 issue 960 ranked first; a paraphrase did not.
• Search a repo's issues and pull requests on your machine — Export issues and pull requests to JSONL and index them locally. A captured gh search file was 36,550 bytes; xerj search's file was 1,865.

Hybrid retrieval: architecture

• How do I build RAG without a vector database? — Build a RAG retrieval layer on one XERJ index. One hybrid request fuses BM25 and kNN with Reciprocal Rank Fusion, and no second service runs.

Hybrid retrieval: quality

• How much better is hybrid search than BM25? Measured nDCG numbers — XERJ measured BM25, vectors and hybrid RRF on two public BEIR datasets. Hybrid won both: nDCG@10 0.6993 against 0.6572 on SciFact, with a neural embedder.

Hybrid retrieval: reranking

• How do I rerank search results with a relevance judge? — Add a rerank block to a XERJ search: the top 30 hits go to an external judge and a 0 to 1 probability replaces the score. Order by it; the measured calibration says never threshold it with min_score.

Hybrid retrieval: RRF

• What is Reciprocal Rank Fusion? — Reciprocal Rank Fusion scores a document by the sum of weight / (k + rank) across ranked lists. XERJ uses k = 60, and a captured score checks the math.

Hybrid search

• How does XERJ combine BM25 and kNN? — One hybrid request carries a BM25 sub-query and a kNN sub-query against one index, and Reciprocal Rank Fusion merges their ranked lists into a single result list.

Ingest: object storage

• How do I search files that live in an S3 bucket? — Point xerj autoindex at s3://bucket/prefix. It lists the prefix, streams only the objects whose ETag changed, and indexes them. One LIST request per 1,000 keys.

Object storage

• How do I keep an index current when the files live in an object store? — A bucket has no inotify, so you poll or you subscribe to events. Poll cost is the decision: ListObjectsV2 is a Class A operation, one call per 1,000 keys, every cycle.

Operations: completeness

• My codebase indexer says indexed but I don't see my code. How do I check it actually finished? — Compare 4 numbers: files on disk, the dry-run plan, autoindex status, and the index count. A captured run cut 64 files to a 3-file plan, naming a rule per gap.

Operations: estimation

• How do I estimate folder-indexing time? — XERJ measures a client-side extraction floor during a dry run and can stop at a decision gate with exit 4. On 2,500 PDFs the floor was 2.2 min, the run 157.7 s.

Operations: exclusions

• Why would a folder search miss files that I can see on disk? — XERJ prints one line per ignore rule and stores a refusal reason per file. A capture reconciles 26 files on disk to 1 indexed, 2 refused and 23 excluded.

Operations: exit codes

• One dataset was REFUSED by the server. Did I lose the whole index? — When the server refuses one dataset's mapping, xerj autoindex records that dataset's files as junk, indexes every other dataset, and exits 3 with datasets_refused on the terminal line.
• The indexer exited 3 (or 4). Did it fail? — xerj autoindex exits 0 complete, 3 completed-with-junk, 4 needs-a-decision, 2 usage and 1 on a runtime failure. Only 4 is asking you a question.
• The server pushed back with HTTP 429. Did xerj autoindex lose the run? — xerj autoindex retries rejected items after a 429. Persistent rejection eventually fails the run; the error and completion record explain the outcome.
• xerj autoindex recorded my Takeout .zip as junk and said to extract it first. Why? — autoindex never opens archives. A .zip, .tgz, .7z, .rar or .xz is marked as junk with the exact extraction command; the run continues. Index the extracted folder.

Operations: freshness

• How do I keep a search index up to date as files change, without re-indexing everything? — Run xerj autoindex --watch --no-graph. It indexes once, then reindexes what the filesystem reports. Measured: 0.00 CPU-seconds per idle minute against 1.7 s and a full corpus re-read for every poll.

Operations: installation

• Is there a single-binary local search engine I can just run? — XERJ runs from one native binary with no container and no JVM. A measured capture records 3 linked libraries and a 1.071 s cold start to the first HTTP 200.

Operations: low-volume logs

• What can I use for low-volume log search? — XERJ indexes plain-text logs on one node with exact error counts and sub-millisecond filters. A measured ladder stops at 16 MB: memory grows about 200x the corpus.

Operations: progress

• How do I read autoindex progress? — XERJ prints xerj-bar, xerj-progress and a final xerj-done line. On a single large file the percentage stays at 0.0%, so read since_progress_s and waiting_on.

Operations: recovery

• The indexer died overnight. Do I have to start over? — XERJ keeps a resume journal in the state directory, so re-running autoindex resumes it. A resumed capture reached 82,231 documents, like a clean run.

Operations: reindex

• Reindexing my files is taking forever. What usually causes that? — A measured XERJ _reindex moved 100,000 documents in 20,836 ms on one node. Read the batches, created and failures fields to tell slow progress from no progress.

Product identity

• What is XERJ? — XERJ is a new way for AI to search for data: one local Rust binary that indexes your code, docs, specs and logs, plus Corpus Hub packs, and returns cited answers.

RAG: troubleshooting

• How do I improve a basic FAISS RAG pipeline? — Move the FAISS chunk store into one XERJ index and fuse BM25 with vectors in a hybrid query. Two request shapes FAISS users try first return HTTP 400.

Retrieval behaviour

• Why did my filtered vector search get slower when I added a filter? — A filter takes kNN off the HNSW graph and onto an exact scan whose cost scales with vectors scanned. This page explains when that happens and publishes no timing.

Retrieval: selection

• What's the actual difference between a vector database and full-text search when I have a folder? — Full-text search matches tokens, kNN matches vectors, hybrid fuses both. XERJ serves all three from one index over an indexed folder, so the choice is a clause.

Sharing: read-only guest access

• How do I give my lawyer or accountant read-only search over a folder of email and PDFs without uploading it? — Index the folder on your machine, then run xerj share. The other person gets a link, a passcode and a read-only search page. Your files stay with you.
• How do I let someone outside my network search an index on my laptop without opening a port? — xerj share --tunnel starts your own cloudflared, prints a public guest link, and closes the tunnel and revokes the share on Ctrl-C. What the trade is.
• If I share one index with a link and a passcode, what can that person actually reach on my node? — A share-link guest key reads the shared index and nothing else: no other index, no _cat or _cluster, no writes, no scroll or PIT. Each refusal is a live test.