Deep Research Systems: A From-First-Principles Technical Reference

Date: 2026-08-01 Scope: What “deep research” / agentic-research systems are architecturally, independent of any vendor product, with each concept mapped onto Vox’s current (partially stubbed) pipeline. This document complements the product-specific competitive audit and code-verification pass produced alongside it. Browse Architecture SSOTs in the Starlight sidebar, or start at contributor-hub.

Deep research systems are LLM-driven pipelines that turn an open-ended question into a cited, evidence-grounded report through iterated retrieval, verification, and synthesis — rather than a single-shot RAG answer. As of 2026 the field has converged on a recognizable reference architecture: planner → multi-hop hybrid retriever → reflection/correction loop → claim verifier → novelty/redundancy filter → confidence gate → grounded synthesizer, evaluated against benchmarks like BrowseComp-Plus and DeepResearch Bench. This document explains each stage from first principles and maps it onto Vox’s existing pipeline in vox-research-shim, vox-research-events, vox-search, vox-scientia, and vox-publisher.

1. Query Planning & Decomposition

What it is and why it matters. A single embedding query cannot cover a multi-faceted research question — “compare X and Y across dimensions A, B, C” needs several retrieval passes, each targeting a narrower sub-question. Decomposition improves recall (each sub-query hits a tighter region of the corpus) and lets downstream stages track which claims answer which part of the original ask. 2026 surveys describe deep research systems as composed of distinct planner, query, retriever, and writer roles, with the planner handling task decomposition and subgoal scheduling (Deep Research: A Survey of Autonomous Research Agents). Products differ mainly in how much decomposition is exposed to the user: some (e.g. Gemini Deep Research) surface the full plan for edit-before-execute, trading latency for alignment (Zylos: Deep Research Agent Architectures).

How it’s implemented. The dominant pattern is an LLM call with a constrained output schema ({"subqueries": [...]}), temperature near 0, capped at a small N (typically 3–8) to bound fan-out cost, followed by deduplication and truncation. Recent RL-trained planners (DeepPlanner) treat planning as a separately optimized skill with advantage-shaped rewards over multi-turn rollouts rather than a fixed prompt, since planning quality dominates downstream benchmark scores (DeepPlanner).

Vox mapping — has it. crates/vox-research-shim/src/research/planner.rs::decompose_query_with_config does exactly this: an LLM cascade call constrained to JSON schema {"subqueries": [...]}, clamped to max_subqueries.clamp(1, 12) (default 6), with case-insensitive dedup and a passthrough_plan fallback (single-subquery plan, flagged planner_degraded: true) when the LLM call or JSON parse fails. This is a solid, production-shaped implementation of the pattern — not a stub, unlike several downstream stages.

2. Multi-Hop Hybrid Retrieval

What it is and why it matters. Sparse lexical retrieval (BM25) is precise for exact terms, IDs, and rare vocabulary but blind to paraphrase; dense embedding retrieval captures semantic similarity but can miss exact-match signals. Neither alone is sufficient — 2026 production guidance is explicit that BM25 excels at exact-match queries while dense retrieval handles conceptual queries, and that fusing them raises NDCG over either alone (Hybrid Search: BM25, Vector & Reranking Reference 2026).

How it’s implemented. The standard pipeline queries sparse and dense indexes independently (top 50–100 each), then merges with Reciprocal Rank Fusion: score(d) = Σ_lists 1/(k + rank(d)), with k≈60 as the standard smoothing constant so top-ranked-in-one-list documents don’t dominate documents ranked well in multiple lists. RRF operates purely on rank position, sidestepping the problem that BM25 and cosine-similarity scores live on incompatible scales. On WANDS, a tuned hybrid setup reaches 0.7497 NDCG, a 7.4% lift over BM25 or dense alone, and recall@10 moves from 78% to 91% when reranking is layered on (Building a Production-Ready Hybrid Retrieval System). Re-querying (“multi-hop”) is triggered when evidence quality (below-threshold fused score, contradictions, low domain diversity) signals the first pass was insufficient — this is the province of CRAG, covered next.

Vox mapping — has it. crates/vox-search/src/rrf.rs::rrf_merge_line_lists implements textbook RRF with a tunable rrf_k (documented as “often ~60”), merging arbitrary ranked line-lists keyed by a stable dedup key (rrf_dedup_key, recognizing [chunk:], [repo:], [node:], [tantivy:], [qdrant:] prefixes). The sparse side is vox-search/src/lexical_tantivy.rs (tantivy/BM25), the dense side is vox-search/src/vector_qdrant.rs (Qdrant) and vox-search/src/memory_hybrid.rs; vox-search/src/web_dispatcher.rs and tavily_research.rs add live web retrieval. Multi-hop re-querying is driven by crates/vox-research-shim/src/research/orchestrator/web_gather.rs::gather_web_hits_for_plan, which loops up to policy.web_search_max_hops rounds, calling CragRouter::should_continue each round.

3. Iterative Reflection Loops (CRAG, ReAct, Reflexion)

What it is and why it matters. These are three related but distinct patterns for deciding “do I have enough evidence, or should I act again?” ReAct interleaves reasoning traces with tool calls in a single loop (Thought → Action → Observation), letting the agent decide dynamically what to retrieve next based on what it just saw (Agentic Loops: From ReAct to Loop Engineering). CRAG (Corrective RAG) adds a lightweight evaluator that scores retrieved evidence quality and triggers a single corrective action (query rewrite, web fallback, knowledge refinement) before generation — a one-shot correction, not an open-ended loop. Reflexion goes further: draft an answer, identify what’s missing via self-critique, retrieve to fill gaps, revise, repeat — a verbal-reinforcement-learning loop that iterates until the answer is judged complete (Reflexion Agentic RAG). 2026 production guidance frames the choice as layered: “start with ReAct, add Reflexion when you need self-correction… CRAG does a single correction” (MarsDevs: Agentic RAG 2026 Production Guide).

How it’s implemented. CRAG evaluators are typically a lightweight classifier or LLM-judge that buckets retrieved evidence into confident/ambiguous/incorrect and picks a corrective branch; Reflexion loops store natural-language self-reflections in an episodic buffer re-injected into the next attempt’s prompt context rather than doing gradient updates. Stopping conditions combine a quality threshold with a hop budget to bound cost.

Vox mapping — has it (CRAG); partial (Reflexion-style self-check). crates/vox-search/src/crag.rs::CragRouter is a direct CRAG implementation: should_continue(current_quality, target_quality, hops_remaining) gates continuation, and expand_queries_from_partial_evidence generates refined sub-queries via two heuristics — (a) regex-extracted capitalized concept terms from top-5 snippets not already in the original query, and (b) gap-oriented follow-ups when hits show potential_contradiction or average score < 0.55. expand_queries_with_llm_or_heuristic prefers LLM-generated refinements when available, falling back to the regex heuristic. The loop is driven from web_gather.rs with CRAG_STOP_QUALITY and policy.web_search_max_hops. Separately, stages.rs::run_self_verification implements a Chain-of-Verification (CoVE)-style pattern (not full Reflexion): generate up to 5 yes/no verification questions from the draft answer, answer each from retrieved context only, and flag critical_inconsistency if more than half come back “no”/“unknown.” This is single-pass self-checking, not an iterate-until-satisfied Reflexion loop with revision — a real gap relative to full Reflexion.

4. Claim Extraction & Verification

What it is and why it matters. Once evidence is gathered, individual factual assertions (“claims”) need per-claim entailment classification against evidence, not just a holistic “does this look plausible” pass — this is what lets a system cite precisely and flag contradictions. The canonical taxonomy comes from FEVER/SciFact-style fact-verification: SUPPORT / REFUTE (Contradict) / NOT ENOUGH INFO, cast as a textual-entailment (NLI) task over (claim, evidence) pairs (SciClaims; Scientific Claim Verification with Fine-Tuned NLI Models).

How it’s implemented. Production systems increasingly use small, distilled, calibrated fact-checkers rather than full LLM judges for cost — MiniCheck and its descendants (Bespoke-MiniCheck-7B, FactCG, VeriFastScore) match GPT-4-level faithfulness scoring at a fraction of the cost by fine-tuning compact NLI models on synthetic claim-evidence pairs. Systems increasingly go beyond a single scalar to typed error taxonomies.

Vox mapping — has it, with a documented taxonomy gap. crates/vox-research-shim/src/research/verifier.rs implements verify_claims_with_config: for each claim, an LLM cascade call (temperature 0, JSON-schema-constrained) classifies against up to 8 evidence snippets, producing a Verdict (Supported | Contradicted | Contested | Unverified) with confidence, an abstain_threshold (default 0.5), and per-span EvidenceSpans typed Supporting | Contradicting | Background. The file’s own doc comment flags that this taxonomy is a legacy compatibility shim, not the canonical one: the SCIENTIA plan specifies SciFact’s Support/Contradict/NotEnoughInfo/Abstain, and reconciling to it is explicitly deferred to “Phase 1’s vox-claim-extractor integration.” VerifierConfig.nli_model_id is a placeholder wired to a FALLBACK_NLI_MODEL_ID registry constant — Vox’s design already anticipates swapping the current LLM-judge verifier for a dedicated NLI/MiniCheck-style model, but that swap hasn’t landed. crates/vox-scientia/ is the crate intended to own claim extraction proper.

5. Novelty / Redundancy / Information-Gain Scoring

What it is and why it matters. Without dedup, an agent burns its hop/token budget re-confirming the same fact from restated sources, and a final report reads as padded rather than comprehensive. Deduplication needs to operate at multiple granularities: exact/near-exact text (shingling/MinHash), semantic near-duplicates (embedding similarity), and — for research-worthiness specifically — novelty relative to prior published work, not just relative to this session’s evidence.

How it’s implemented. The standard toolkit layers: exact hash dedup → MinHash/LSH on n-gram shingles with Jaccard-similarity thresholds (~75% typical) for near-duplicates → embedding-based semantic dedup via an ANN index for paraphrase-level redundancy (MinHash LSH in Milvus). “Information gain” gating means scoring a new candidate fact by how much new shingle/embedding mass it contributes versus content already accepted, not merely whether it’s a binary duplicate.

Vox mapping — has two independent implementations at different maturity, not yet unified. (1) crates/vox-search/src/novelty.rs::NoveltyScorer — a lightweight per-session scorer: 4-gram character shingles hashed with FNV-1a into a HashSet; score() returns the fraction of a new hit’s shingles not already seen, and accept() commits them. Jaccard-adjacent but simpler than true MinHash (no LSH bucketing), fine at session scale. (2) crates/vox-publisher/src/scientia_prior_art.rs + crates/vox-research-events/src/schema_types.rs::NoveltyEvidenceBundle — a considerably more sophisticated cross-corpus novelty pipeline: federated queries against OpenAlex, Crossref, and Semantic Scholar, producing NormalizedHits with both lexical_score and semantic_score, bucketed NoveltyRecencyBucket (VeryRecent/Recent/Stale), and an OverlapSummary — used to assess whether a research finding is prior-art-novel before self-publication, not to dedup retrieval hits mid-session. Separately, crates/vox-similarity/ (signature.rs, index.rs) implements genuine MinHash signatures, currently consumed by vox-skill-discovery and vox-orchestrator’s similarity services, not yet by vox-search’s per-session novelty scorer — a natural, low-risk upgrade path if novelty.rs’s FNV shingle-set approach needs LSH-scale performance.

6. Cross-Encoder Reranking

What it is and why it matters. RRF fusion combines independently computed rank signals — it never lets the query and document interact except through separate encoders. A cross-encoder jointly encodes (query, document) as one sequence and outputs a single relevance score, capturing token-level interactions bi-encoders and BM25 both miss. This is why the 2026 consensus production stack is explicitly “BM25 + dense + RRF + cross-encoder re-ranking,” not RRF alone — the fused top-k (typically top 50) is re-scored by a cross-encoder before truncation to the final context window (Hybrid Search and Re-ranking in Production RAG 2026).

How it’s implemented. A small transformer (e.g., a BGE-reranker or Cohere Rerank-class model) scores each of the top-N fused candidates against the query in one forward pass per pair; too expensive over the full corpus, cheap enough over top-50.

Vox mapping — lacking. A repo-wide search for reranking/cross-encoder logic in vox-search returns no matches. Vox’s retrieval pipeline stops at RRF-fused hits; there is no second-stage cross-encoder pass before hits reach the CRAG loop, verifier, or synthesizer. This is one of the more concrete, well-scoped gaps relative to 2026 production practice.

7. Synthesis & Citation Grounding

What it is and why it matters. The final report must attribute every material claim to a real, checkable source — LLMs left unconstrained will fabricate citations. 2026 work decomposes “citation hallucination” into distinct failure modes needing distinct fixes: factual, grounding, citation (existence/relevance/temporality), and reasoning errors (Citation Grounding: Detecting and Reducing LLM Citation Hallucinations). Reported citation-hallucination rates in production legal-domain systems run 13–21% even with grounding discipline.

How it’s implemented. Mitigations layer: (a) grounding-discipline system prompts instructing “cite only from provided evidence, using these exact markers”; (b) structured citation schemas enforced at the output-parsing layer; (c) runtime registry checks that the cited URL/DOI/chunk-id actually resolves and exists in the evidence set; (d) sampled semantic-entailment checks on (claim, cited passage) pairs, reusing the NLI verifier from §4 as a post-hoc citation auditor.

Vox mapping — has it, template fallback included; missing post-hoc audit. crates/vox-research-shim/src/research/orchestrator/stages.rs::synthesize_answer_with_llm builds numbered evidence blocks ([1] title / URL / snippet) and instructs the synthesis LLM to cite inline matching evidence numbers — structural grounding-by-construction. It appends claim-verdict summaries from §4’s verifier into the synthesis context, so contradicted/unverified claims are visible to the writer model. A deterministic synthesize_answer_template fallback (no network call) emits a “Verification Status” section with per-claim verdict icons and a mechanically-built “Citations” section — sidesteps hallucination entirely for the no-LLM path. What’s missing relative to 2026 SOTA: no post-hoc registry check that a cited [N] marker in the LLM-generated text actually maps to real evidence, and no sampled semantic-entailment re-check between synthesized sentences and their cited spans — the verifier runs on planner-extracted claims, not on the synthesized prose itself.

8. Confidence / Quality Gating

What it is and why it matters. A single scalar like “citation count” is a weak proxy for report quality. Production systems fuse multiple signals (retrieval confidence, claim-support ratio, source diversity, contradiction rate) and increasingly build in explicit abstention: refuse or downgrade rather than answer confidently on thin evidence (Engineering the RAG Stack). 2026 work frames multi-signal fusion as outperforming single-threshold gating for hallucination reduction, with some approaches requiring multiple independent signal families to agree (conjunctive gating) before granting confidence (CalVerT).

How it’s implemented. A gate combines: retrieval-hit presence (hard zero-out if no hits), claim-support ratio from the verifier, source/domain diversity count, and sometimes calibrated verbal-confidence self-reports — fused into a score that routes to different response tiers.

Vox mapping — has a documented stub in the main loop, plus a more mature parallel implementation elsewhere. crates/vox-research-shim/src/research/gate.rs is explicitly self-labeled Phase 0a STUB: score_with_config fuses citation_score (0.35 weight), claim_support_score (0.30, from the verifier’s supported_claim_count), distinct_domain_count-based diversity_score (0.20), and a hardcoded retrieval_score = 1.0 (0.15) — already a multi-signal weighted fusion, not literally pure citation count, but the module comment concedes it’s a placeholder pending fusion of contradiction ratio (notably absent from the current formula despite the verifier producing contradicting_count per claim). no_retrieval_hits triggers a hard zero (abstention-shaped), and ConfidenceSignal::routing_tier_for maps the score to Direct/Light/DeepResearch tiers — exactly the “escalate to deeper research on low confidence” pattern from the literature. Separately and more maturely, crates/vox-research-events/src/schema_types.rs::WorthinessSignalsV2 implements genuine conjunctive multi-signal gating for research-finding publication: hard_gate/soft_gate/diagnostic signal buckets, each item carrying passed: bool, score: f32, and a reason_code string plus next_actions — closer to the CalVerT-style auditable, reason-coded gating pattern than gate.rs is, but it currently gates publication worthiness, not the retrieval-confidence routing decision in the main research loop. Unifying these two gates (or porting WorthinessSignalsV2’s reason-coded structure into gate.rs) is a concrete, low-risk architectural improvement visible directly from the code.

9. Evaluation Harnesses for Deep Research Systems

What it is and why it matters. Deep research is hard to eval because retrieval quality and reasoning/synthesis quality are confounded. 2026’s two leading benchmarks disentangle these differently: BrowseComp-Plus (ACL 2026) fixes retrieval to a ~100K-document human-verified corpus rather than the live web, isolating agent reasoning/query-formulation quality from search-engine variance and letting retrievers be swapped to isolate their contribution. Reported numbers: with BM25 retrieval, Search-R1 gets 3.86% accuracy vs. GPT-5’s 55.9%; pairing GPT-5 with a Qwen3-Embedding-8B retriever pushes accuracy to 70.1% — retriever quality alone swings accuracy ~15 points even with a fixed strong LLM (BrowseComp-Plus; ACL Anthology). DeepResearch Bench (100 PhD-level tasks across 22 fields, expert-authored) evaluates the report end-to-end — DeepResearch Bench II (Feb 2026) scores generated reports against 9,430 fine-grained binary rubrics spanning information recall, analysis quality, and presentation (DeepResearch Bench; GitHub). A complementary framing question is asked directly in “How Far Are We from Genuinely Useful Deep Research Agents?”.

Vox mapping — lacking. crates/vox-search/src/evaluation.rs exists but its scope wasn’t confirmed against these benchmark shapes in this pass. Nothing in the reviewed crates resembles a BrowseComp-style fixed-corpus retriever-isolation harness or a DeepResearch-Bench-style rubric-based report scorer. Vox’s gate.rs/verifier.rs self-tests check unit-level correctness, but there’s no end-to-end benchmark verifying the composed pipeline (planner → CRAG → verifier → gate → synthesis) produces reports that score well on recall/analysis/presentation rubrics, nor a fixed-corpus mode for isolating retriever contribution from LLM reasoning contribution. Building even a small internal rubric-based harness (borrowing DeepResearch Bench’s binary-rubric methodology against a handful of Vox-specific research tasks) is a natural next step given the pipeline is otherwise fairly mature.

Summary Table

#	Concept	Vox status	Primary file(s)
1	Query planning & decomposition	Has it	`vox-research-shim/src/research/planner.rs`
2	Multi-hop hybrid retrieval (BM25+dense+RRF)	Has it	`vox-search/src/rrf.rs`, `lexical_tantivy.rs`, `vector_qdrant.rs`, `web_dispatcher.rs`
3	CRAG loop	Has it	`vox-search/src/crag.rs`, `vox-research-shim/.../web_gather.rs`
3	Reflexion-style iterative self-revision	Partial (CoVE-style single-pass check only)	`vox-research-shim/.../stages.rs::run_self_verification`
4	Claim extraction & NLI verification	Has it, taxonomy not yet reconciled to SciFact	`vox-research-shim/src/research/verifier.rs`
5	Novelty/redundancy scoring	Has it (two unconnected implementations)	`vox-search/src/novelty.rs`; `vox-publisher/src/scientia_prior_art.rs` + `vox-research-events`
6	Cross-encoder reranking	Lacking	—
7	Citation-grounded synthesis	Has it, no post-hoc citation audit	`vox-research-shim/.../stages.rs::synthesize_answer_with_llm`
8	Multi-signal confidence gating	Partial/stub in main loop; mature version exists elsewhere	`vox-research-shim/src/research/gate.rs` (stub); `vox-research-events/src/schema_types.rs::WorthinessSignalsV2` (mature)
9	Evaluation harness	Lacking	—

Deep Research Systems: A From-First-Principles Technical Reference

Deep Research Systems: A From-First-Principles Technical Reference

1. Query Planning & Decomposition

2. Multi-Hop Hybrid Retrieval

3. Iterative Reflection Loops (CRAG, ReAct, Reflexion)

4. Claim Extraction & Verification

5. Novelty / Redundancy / Information-Gain Scoring

6. Cross-Encoder Reranking

7. Synthesis & Citation Grounding

8. Confidence / Quality Gating

9. Evaluation Harnesses for Deep Research Systems

Summary Table

Sources