# Hybrid retrieval: search broadly, rerank precisely.

A practical retrieval pipeline using permission filters, BM25, vectors, fusion, deduplication, reranking and separate evaluation.

Version 1.0 · 2026-07-29 · 9 min

## Short answer

Use permission and product-scope filters before retrieval. Run lexical search for exact terms and semantic search for meaning, merge their candidates, remove duplicates and rerank a small set against the actual question. Send only the best evidence to the model. Evaluate retrieval, relevance, faithfulness and final task outcome separately. Hybrid retrieval is not automatically better: it earns its complexity only when representative queries show that each stage fixes a real failure mode.

## Key points

- Apply access filters before candidate generation.
- Use lexical and semantic search for complementary failures, not fashion.
- Evaluate every stage separately so answer quality does not hide retrieval defects.

## Product language contains both exact identifiers and fuzzy intent

Users quote error codes, field labels and acronyms that lexical search handles well. They also describe goals in language that does not appear in the documentation, where semantic retrieval can help. A single retriever often misses one of these forms.

The solution is not to concatenate every result. Fusion and reranking should produce a small, diverse evidence set. Otherwise the generator must resolve duplicate, weak or contradictory chunks at higher cost.

## A production retrieval pipeline

Each stage should have an observable input, output and reason to exist.

### 1. Filter

Apply tenant, product version, role, locale and source-status filters before search. Security is not a reranking feature.

### 2. Generate lexical candidates

Use BM25 or an equivalent for exact terms, identifiers, rare vocabulary and clear keyword matches.

### 3. Generate semantic candidates

Use embeddings for paraphrases and conceptual similarity. Keep model, chunking and version information reproducible.

### 4. Fuse and deduplicate

Merge ranks with a stable method, collapse repeated chunks and retain source diversity when it helps the task.

### 5. Rerank

Score a bounded candidate set against the full question and product context. Avoid an expensive reranker when the first stage is already decisive.

### 6. Evaluate and cache

Measure retrieval recall and precision, then answer faithfulness and task completion. Cache stable stages with explicit invalidation.

## Examples

### Exact code, broad explanation

BM25 finds the error identifier while semantic search finds the procedure described with different words. Reranking keeps the relevant pair.

### Role-specific guide

Metadata filtering removes guides for other roles before either retriever runs.

### Repeated documentation

Deduplication prevents the same release note copied across pages from dominating the context.

## Common failure modes

- Applying access filters after candidate text has entered model context.
- Using only vector search for exact codes and product labels.
- Increasing top-k until the correct chunk appears, then flooding generation.
- Evaluating only final answers and missing a fragile retrieval stage.
- Treating an LLM-based evaluator as ground truth without calibrated examples.

## Checklist

1. A representative query set includes exact terms, paraphrases and negatives.
2. Metadata filters are deterministic and tested.
3. Lexical and semantic candidate sets are logged in redacted form.
4. Fusion and deduplication rules are stable.
5. Reranking operates on a bounded set.
6. Retrieval and generation metrics are reviewed separately.
7. Source changes and caches have an invalidation policy.

## Limitations

- Hybrid search adds infrastructure, tuning and evaluation work.
- Public benchmark gains may not transfer to a specialised product corpus.
- Rerankers can still prefer fluent but wrong context and need domain evaluation.

## Sources

1. [The Probabilistic Relevance Framework: BM25 and Beyond](https://doi.org/10.1561/1500000019) — Robertson and Zaragoza, 2009; type: research.
2. [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://aclanthology.org/2022.naacl-main.272/) — Santhanam et al., 2022; type: research.
3. [RAGAS: Automated Evaluation of Retrieval Augmented Generation](https://aclanthology.org/2024.eacl-demo.16/) — Es et al., 2024; type: research.
4. [QMD: query Markdown with hybrid search](https://github.com/tobi/qmd) — Tobi Lütke, 2026; type: field-practice.

## Practical questions

### Do we always need vectors?

No. Start with a baseline. A small, well-labelled corpus with exact product terms may be served well by lexical search.

### How large should the final context be?

As small as possible while preserving the evidence needed for the answer. Determine it from evaluations, not a universal token count.

### Where does QMD fit?

QMD is a useful field implementation of hybrid Markdown search. It is an engineering reference, not scientific proof that one pipeline fits every corpus.

---

Canonical: https://scoperail.fokalabs.io/en/guides/hybrid-retrieval-and-reranking
Author: Ramzi Laieb, FokaLabs