Hybrid search puts keywords and meaning in one ranking
BM25 handles exact terms, vectors handle intent and a reranker settles the order. We build hybrid retrieval that handles product codes, people's names and plain-language questions alike.
Pure semantic search is excellent at paraphrase and poor at exactness. Hybrid search exists for corpora where both matter.
You will see the gain when a meaningful share of queries contain things an embedding model blurs: part numbers, ticket IDs, error codes, surnames, legal clause references, or a product name that differs from a near-identical one by a single character. You will also see it on mixed-language content, long-tail technical vocabulary, and documents where a single rare term is the whole point.
Hybrid pays off when
Users search by SKU, code, name or exact phrase as well as by description
The corpus mixes languages or contains domain jargon the model has rarely seen
Recall on rare terms matters: compliance, engineering, support knowledge bases
A keyword engine already exists and you want to add meaning without losing it
Plain vector search is enough when
Content is prose in one language and queries are plain questions with no codes or identifiers
The corpus is small and a reranker can read the whole candidate set
Nobody on the team can own a second retriever and its index
Keyword side
BM25
Scores documents by how often the query terms appear, discounted for common words and long documents. It is exact and fast, and blind to synonyms.
Meaning side
Dense vectors
Embeds query and passages, returns the nearest neighbours. Handles paraphrase and intent, but blurs codes and negation.
Final order
Cross-encoder reranker
Reads the query and each candidate together and scores the pair. Slow per document, so it only sees the merged shortlist.
How it works
Two retrievers in parallel, one fused list, one reranker
The query "4471 won't charge" reaches both retrievers at once: keyword search finds the code, vector search finds "battery not charging". RRF merges the two lists, the reranker orders the shortlist, and five passages go on.
Each stage narrows the set. Retrievers are cheap and return many candidates. Fusion is free. The reranker is expensive per document, so it only sees the merged shortlist. This shape keeps latency flat while recall and precision both improve.
Reciprocal rank fusion
Merging two lists without comparing their scores
BM25 produces scores like 12.7; vector search produces similarities like 0.83. They are not on the same scale, and normalising them is fragile. Reciprocal rank fusion sidesteps the problem by ignoring scores and using positions.
Each list gives every document it contains a contribution of 1 ÷ (k + rank). A document found by both lists collects two contributions; a document found by one collects one. Add them up, sort by the total, and that is the fused order. The constant k dampens the gap between rank one and rank two; 60 is a widely used default and rarely needs changing.
The effect is intuitive. A document that is moderately good in both lists beats a document that is first in one and absent from the other. That is exactly what you want when one retriever is confidently wrong.
Document
BM25 rank
Vector rank
RRF score (k = 60)
Fused rank
Returns policy
3
1
1/63 + 1/61 = 0.0323
1
Refund timelines
2
4
1/62 + 1/64 = 0.0318
2
Model 4471 spec sheet
1
not in list
1/61 = 0.0164
3
Worked example with three documents. The spec sheet topped the keyword list but appears nowhere in the vector list, so two moderate placements beat it.
Reranking
The step that fixes "close but wrong"
Retrievers compare a query to each document separately. A cross-encoder reads them together, which is why it catches negation, qualifiers and the difference between two similar products.
What it does
Takes each (query, passage) pair, produces a relevance score, and reorders the shortlist. It judges the candidates the retrievers found.
What it costs
One model call per candidate. Rerank 20–50 passages rather than 500. For chat-style interfaces this fits comfortably inside the time the LLM takes to answer.
Where it comes from
Hosted rerank APIs from several model providers, or open-weight cross-encoders you run yourself. We choose based on data residency and latency, then confirm on your test set. See reranking.
Product features change often; we confirm the current behaviour of each engine during the selection sprint. The vector database selection guide covers the wider decision.
Process
How we build and tune hybrid retrieval
Baseline (week 1)
Label 100–300 real queries. Score the existing search, or a pure vector prototype, so every later change has a number to beat.
Two retrievers (weeks 2–3)
Stand up keyword and vector retrieval over the same chunks, with shared document IDs and metadata.
Fuse and rerank (weeks 3–4)
RRF first, then a reranker over the shortlist. Compare against weighted fusion on the test set.
Tune (weeks 4–5)
Candidate counts, filters, chunk size and query rewriting, each judged by recall@k and MRR.
Ship and monitor (weeks 5–6)
Shadow traffic, latency budget, dashboards and a feedback path from real queries into the test set.
Hybrid retrieval is also the retrieval layer we recommend for most enterprise RAG builds, where a missed passage becomes a wrong answer. If you are deciding how far to go, the RAG explained guide covers the trade-offs, and evaluating LLM outputs covers the measurement side.
Hybrid search runs a keyword retriever (usually BM25) and a vector retriever on the same query, merges the two result lists, and often passes the merged list through a reranker. It keeps exact matching for codes and names while adding meaning-based recall. See the glossary entry.
Is reciprocal rank fusion better than weighting the scores?
RRF is the safer default because it only uses ranks, so you never have to reconcile BM25 scores with cosine similarities. Weighted score fusion can edge ahead when you have a good test set to tune the weight on. We start with RRF and only move to weighting when the evaluation shows a gain.
Does hybrid search make queries slower?
Two retrievers run in parallel, so the retrieval stage adds little. The reranker is the cost: it scores each candidate with a model. Keeping the reranked set to 20–50 candidates keeps latency acceptable for interactive search in most set-ups.
Can we do hybrid search inside PostgreSQL?
Yes. Postgres full-text search provides the keyword side and pgvector the vector side. Both run in one SQL statement and the fusion can be done in SQL or in the application. It is the lowest-effort route when Postgres already holds the content.
When is plain vector search enough?
When your corpus is prose written in one language, queries are natural-language questions, and nobody searches by code, name or exact phrase. In that case a vector retriever plus a reranker usually matches hybrid on the metrics, with less to operate.
Talk to us about your AI project
Tell us what you are trying to do, which systems are involved and what is in the way. We reply within one business day with an opinion and a suggested next step.