Recommendations
“People who read this also read” from content similarity alone, no click history required on day one.
Recommendation engine
Vector Search
Vector search that finds meaning, not just keywords
We design, build and tune semantic and hybrid search on pgvector, Milvus, Pinecone, Qdrant or Weaviate, and prove the gain on your own queries before it goes live.
- Hybrid search with reranking
- Milvus and pgvector in our own production
- Evaluated on your queries, not a demo set
What it is
Search by what people mean
Keyword search matches strings. Vector search matches meaning, so “can I send it back?” finds the returns policy even though no word overlaps.
An embedding model turns each piece of content into a vector, a long list of numbers that encodes its meaning. Similar content ends up near each other. A query is embedded the same way, and the index returns the closest vectors in milliseconds, even across millions of items.
That retrieval layer sits underneath every serious AI knowledge base, but it is useful on its own: site search, product discovery, helpdesk search and internal document search all improve when the engine understands paraphrase.
The craft is in the details: which model, how to split content, when to combine with keywords, which database, and how to prove the result on your own queries. Those are the pages in this section.
1. Embed
Content is chunked and converted to vectors with a model chosen for your language, domain and budget.
2. Index
Vectors plus metadata go into an approximate nearest-neighbour index (HNSW, IVF or similar) in the store that fits your stack.
3. Query
The question is embedded, filtered by metadata or permissions, and the nearest candidates are returned, often fused with a keyword search.
4. Rerank
A cross-encoder re-scores the top candidates against the query so the best few reach the user or the LLM.
Compare
Keyword, semantic or hybrid?
Each approach has a failure mode. Hybrid search exists because production queries contain both natural language and exact tokens, often in the same sentence.
| Criterion | Keyword (BM25) | Semantic (vectors) | Hybrid + rerank |
|---|---|---|---|
| Synonyms and paraphrase | Misses unless words match | Strong | Strong |
| Exact codes, SKUs, names, error strings | Exact | Unreliable | Exact, via keyword leg |
| Typos | Needs fuzzy matching | Partly tolerant | Good with fuzzy keyword leg |
| Multilingual queries | Per-language setup | Multilingual models handle it | Yes |
| Explaining why a result matched | Highlighted terms | Opaque similarity score | Terms plus score |
| Infrastructure | Any search engine or Postgres | Embedding model + vector index | Both, plus a fusion step |
| Best fit | Catalogue codes, logs, legal exact phrases | Help content, FAQs, long-form text | Most production systems |
Deeper treatment on semantic search and hybrid search; definitions in the glossary under hybrid search and reranking.
Beyond chat
Where vector search earns its keep outside a chatbot
Nearest-neighbour search is a general tool. Once embeddings exist, several other problems become cheap.
Deduplication
Find near-duplicate tickets, articles or records that differ in wording but not in substance, before they pollute a knowledge base.
Clean up a knowledge baseClassification and routing
Label incoming emails, tickets or documents by their nearest labelled examples, cheaper and more adjustable than a trained classifier.
Document processingAnomaly detection
Transactions or events far from every known cluster are worth a second look. Distance is the signal.
Fraud and anomaly detection
How we work
A search engagement, start to finish
A retrieval pilot usually takes three to five weeks. Adding hybrid search, reranking and a production index to an existing system takes two to six more, depending on data volume.
Collect real queries
We pull 100 to 300 actual searches or questions and label the documents that should answer each. This set is the yardstick for everything that follows.
Baseline
Measure your current search (or a plain keyword index) on recall@k and MRR, so the improvement is a number, not an impression.
Design
Pick the embedding model and chunking, decide on hybrid fusion and reranking, and choose the vector database for your scale and hosting constraints.
Build and tune
Ingestion pipeline, index, query service with filters, and a tuning loop on the test set. Latency and cost are tracked alongside quality.
Ship and monitor
Go-live with dashboards for query latency, zero-result rate and click-through, plus a re-embedding plan for when content or models change.
Methods
Vector search services
Semantic search
Meaning-based search for sites, catalogues, helpdesks and internal documents.
ExploreHybrid search
BM25 plus vectors, fused with reciprocal rank fusion and reranked.
ExploreEmbeddings strategy
Model choice, chunking, metadata and a plan for re-embedding without downtime.
ExploreVector database selection
A qualitative comparison and a decision tree across five engines.
ExploreDatabases we implement
pgvector
Vectors inside PostgreSQL. The right first choice when Postgres already exists.
ExploreMilvus
Distributed, open-source engine for large scale. We run it in production.
ExplorePinecone
Fully managed and serverless. Fast start, trade-offs to understand.
ExploreQdrant & Weaviate
Open-source engines with strong filtering, hybrid search and managed clouds.
ExploreFree tools
Interactive
RAG cost estimator
One-off indexing, monthly query cost and storage from your document counts, with editable unit prices.
Open the estimator InteractiveVector database chooser
Eight questions about scale, hosting, filtering and ops capacity, then a ranked recommendation with reasons.
Get a recommendation
Related
Go deeper
Guide
Vector database comparison
Long-form comparison of the engines, written for decision makers.
Read GuideChoosing embedding models
Dimension, language, domain, cost and how to test candidates.
Read ServiceAI knowledge base
Put the retrieval layer to work answering questions with citations.
Read ServiceAI data engineering
Pipelines that keep content, metadata and embeddings in sync.
Read
FAQ
Frequently asked questions
What is vector search in plain terms?
Text, images or records are converted into lists of numbers called embeddings, which place similar things close together in a mathematical space. A search embeds the query the same way and returns the nearest neighbours. The result is search by meaning rather than by exact words. See our glossary entry on embeddings.
Do we need a dedicated vector database?
Not always. If you already run PostgreSQL and expect fewer than a few million vectors, pgvector is usually the right start. Dedicated engines earn their place at larger scale, with heavy filtering, multi-tenancy or when you want a fully managed service. The vector database chooser gives a ranked recommendation from eight questions.
Is semantic search better than keyword search?
It is better at paraphrase, synonyms and natural-language questions, and worse at exact codes, names and rare terms. Most production systems use hybrid search, which runs both and fuses the results. Our hybrid search page explains the fusion step in plain words.
How do you measure whether search quality improved?
With a labelled test set of 100 to 300 real queries and the documents that should come back for each. We track recall at k and mean reciprocal rank on every change, so a new embedding model or chunking rule has to prove itself. The method is the same one we use for LLM evals.
Which vector databases do you work with?
PostgreSQL with pgvector, Milvus, Pinecone, Qdrant and Weaviate. We run Milvus and pgvector in our own production systems and have implemented the others for clients. Our selection guide compares them qualitatively, without vendor benchmarks.