Before RAG: check whether retrieval finds the evidence
Use six fictional documents and six queries to check retrieval, source IDs and unanswerable cases with our offline Node.js example; no model is loaded.
Model to check
Nemotron-3.5-Embed-1B-BF16
Nemotron-3.5-Rerank-8B-BF16
Models generate and reason. Agents use tools to work on tasks. A harness manages an agent’s environment. MCP connects tools and data. Skills are reusable task guidance.
Before you start
Prerequisites
Node.js 22+, ZIP extraction and a new writable folder; no GPU, Python, account, key or model. English fixture is not an eight-language retrieval benchmark.
Model / service
No model initially; this example only retrieves literal word matches and calculates metrics.
Agent / runtime
No agent, MCP or server; document text remains data.
Estimated time
20–30 min
Implementation steps
Download and extract the four files. Read README and MIT terms; keep private documents and secrets out.
In that folder check Node.js 22 or newer, then run node retrieval.mjs. No npm packages, model or database are needed.
Compare candidate IDs with relevantIds and read the fixture text. A high score reflects word overlap, not trustworthy evidence.
Run with k=1 and k=2. The backup query has two relevant documents: one correct first hit still means Recall@1 is 0.5.
Inspect both unanswerable queries: one has no hit, the other a false positive. Record that error; candidates are not AI answers or safety certificates.
Save a copy, change one relevantIds entry to missing-id and expect a nonzero input error. Restore the file and confirm normal output.
Next label a small authorized corpus independently. When testing embeddings or rerankers, keep corpus, labels and k fixed and check license, backend and costs separately.
Commands and examples
Not tested by AQI AI
# In the extracted example folder; original synthetic English data only.
node --version
node retrieval.mjs
node retrieval.mjs fixture.json 1
node retrieval.mjs fixture.json 2
# Back up fixture.json, change one relevant ID to missing-id, and rerun.
# Expect a nonzero input error, then restore the original file and rerun.Validation checks
Only the original lexical fixture ran on Windows/Node.js: original, invalid-reference failure, restoration and k=1 checks passed. No embedding, reranking, parsing or RAG-answer tests; one unanswerable false positive is intentional.
Common problems
Missing file: run in the extracted folder. missing-id is an intended input error; restore fixture.json. Recall is not answer correctness; keep negative cases rather than deleting them to inflate metrics.
Cost
Free software; check license and compute costs
MIT example has no fee or API calls; supply your own computer and electricity. Later model/hosting costs require separate checks.
Limits & boundaries
Permissions & risks
The program only reads the selected fixture and prints to the console; no network, upload, port or credential access. Recheck data flow when adding a real service.
Alternatives and next steps
Open-source alternatives may still require paid hardware or model calls.
You can also use a manual table of queries, relevant IDs and retrieved hits without running code. Real models/services need separate cost, license and hardware checks.
Original sources
- Sentence Transformers · Retrieve & Re-Rank ↗
www.sbert.net · · Official documentation
- Sentence Transformers · InformationRetrievalEvaluator ↗
www.sbert.net · · Official documentation
- Nemotron 3.5: separate parsing, retrieval and reranking ↗
huggingface.co · · Official documentation