Milvus consulting for vector search that scales past one server
We run Milvus in production and bring that experience to your deployment: cluster design, index choice, filtering, tenancy and the operations that keep it healthy.
Milvus is an open-source vector database built for large collections and high query throughput. It separates storage from compute, so each role scales on its own. That design is what makes it worth the extra operations.
Collections too large or too busy for a single database server to hold comfortably.
Query load that spikes, where you want to add search capacity without touching ingestion.
Several collections with different index types, filters and latency targets.
GPU budget for very high throughput, or disk-based indexes to keep memory in check.
A platform team that already runs Kubernetes and prefers open source to a managed-only vendor.
We run Milvus in production behind our own AI products, so the advice here comes from operating it rather than from a feature list. We know where the sharp edges are: memory planning for loaded segments, compaction behaviour, upgrade sequencing, and what to monitor before users notice anything.
Signs it is more than you need
A modest corpus and Postgres already in place: start with pgvector.
Nobody available to own a stateful Kubernetes workload.
One application, simple filters, predictable traffic.
A hard requirement for zero infrastructure: consider Pinecone.
How it works
How Milvus is put together
Every write is appended to a streaming log before anything else happens. Everything downstream consumes that log or the object storage it feeds. Object storage is the source of truth; the worker nodes are replaceable.
Writes go to the log first. Data nodes seal segments into object storage, index nodes build indexes beside them, and query nodes load both to serve searches. Fresh rows are read straight from the log, so they are searchable before they are indexed.
What each role does
Proxies at the front validate requests, fan them out and merge the partial results. Data nodes consume the log, group rows into segments and persist each sealed segment to object storage. Index nodes pick up sealed segments and write index files beside them. Query nodes load sealed segments and their indexes into memory, and read the newest rows from the log. A coordinator keeps the metadata, assigns segments and balances the load.
Why that matters to you
You scale the role under pressure and leave the rest of the system alone. More search traffic means more query nodes. A bulk ingest means more data and index nodes for an afternoon. Because object storage holds the truth, a lost worker is replaced rather than recovered. The price is more moving parts: a log broker, etcd, object storage and several node types to size and watch.
Deployment ladder
Start small, step up without re-platforming
The same client code talks to every tier. We start on the smallest one that meets the brief and plan the step up before it is urgent.
Tier
Runs as
Good for
Scaling
Ops burden
Milvus Lite
Embedded library inside a Python process
Prototypes, notebooks, tests, small local sets
None
None
Standalone
One server with Docker: all roles in one process plus etcd and object storage
Small to mid-sized production, one team
Vertical
Low to medium
Distributed
Kubernetes via Helm or the Milvus Operator; each role as its own pods
Large collections, high throughput, high availability
Horizontal, per role
Medium to high
Zilliz Cloud
Managed service from the Milvus maintainers
Teams without platform capacity, or wanting a managed SLA
Managed
Low
Design decisions
Indexes, partitions, filtering and tenancy
Index families, chosen per collection
FLAT gives exact results and is the yardstick. IVF variants cluster the space and can quantise vectors to shrink memory. HNSW builds a graph for low-latency search at high recall. Disk-based indexes serve collections larger than memory, and GPU-accelerated indexes exist for very high throughput. We pick per collection and measure recall against FLAT on a sample before committing.
Partitions and partition keys
Partitions split a collection so a query touches only the slices it needs. A partition key, such as tenant or region, routes rows automatically on insert and narrows searches on read. Combined with scalar filters, this keeps multi-tenant latency flat as tenants are added.
Scalar filtering and hybrid search
Filters on scalar fields run inside the search rather than after it, so candidate sets stay correct even with selective filters. Current releases add sparse vectors and multi-vector hybrid search, so keyword-style and semantic signals can be fused in one request. See hybrid search for when that pays off.
Multi-tenancy at the right level
Milvus can separate tenants by database, by collection or by partition key within one collection. Few large tenants with strict isolation get their own collections; many small tenants share a collection with a partition key. We choose by tenant count, isolation requirement and the privacy obligations attached to the data.
What we deliver
A Milvus deployment you can run after we leave
Architecture and sizing
Tier choice, node counts, memory per query node, object storage and log broker selection, written down with the assumptions.
Collection and index design
Schemas, partition keys, index types and parameters per collection, with measured recall and latency on your data.
Backups you have restored. The Milvus backup tooling copies collections to object storage. A backup counts only after a restore drill into a scratch cluster.
Upgrades in order. Read the release notes, upgrade dependencies first, rehearse on a copy, then roll the cluster. Skipping versions is where outages come from.
Monitoring that watches memory. Milvus exports metrics for Prometheus. We alert on query-node memory, log consumption lag, compaction backlog and load failures, as well as uptime.
Kubernetes hygiene. Resource requests and limits per role, a dedicated node pool for query nodes and persistent volumes for etcd. Pod disruption budgets so maintenance does not drop search.
On-prem, cloud or managed
Milvus runs wherever Kubernetes runs, which makes it a strong fit for data that must stay in your own environment. If residency is not a constraint and platform capacity is, Zilliz Cloud removes most of this list. Our on-prem vs cloud guide walks through the decision for the whole LLM stack.
Process
How a Milvus engagement runs
Week 1
Assess and size
Data volumes, embedding width, query patterns, filter selectivity, tenancy, residency constraints and who will operate the cluster. Output: tier recommendation and sizing sheet.
Weeks 2–3
Stand up and load
Deploy the chosen tier, wire object storage and the log broker, build the ingestion pipeline and load a representative sample.
Weeks 3–5
Index, filter, measure
Index per collection, add partitions and filters, measure recall against FLAT and latency under load. Tune until the targets hold with headroom.
Weeks 5–8
Harden and hand over
Monitoring, backup and restore drill, upgrade rehearsal, runbook and a walkthrough with your platform team. The RAG cost estimator gives you a running-cost view to track against.
Not at first. Milvus Standalone runs on one server with Docker and covers many production workloads. The distributed deployment, which scales each role separately, is designed for Kubernetes. We start you on the smallest tier that meets the brief and plan the step up.
Standalone or distributed: how do we decide?
Standalone when one server's memory holds the loaded segments and the team wants simple operations. Distributed when you need to scale query capacity independently of ingestion, want high availability across nodes, or expect the collection to outgrow one machine.
Should we self-host or use Zilliz Cloud?
Zilliz Cloud, the managed service from the Milvus maintainers, removes the platform work. Self-hosting gives you control over residency, hardware and cost structure. If no one on the team can own a Kubernetes workload, managed is the honest answer. See on-prem vs cloud.
Can we migrate from pgvector to Milvus later?
Yes, and we design for it. Keep embeddings and metadata in a form you can re-load, put retrieval behind a thin interface, and the move is a data migration plus an adapter swap. Re-embedding is only needed if you change the model at the same time.
Does Milvus support hybrid keyword and vector search?
Current releases support sparse vectors alongside dense ones and multi-vector hybrid search, so keyword-style and semantic signals can be fused in one request. The hybrid search page explains the fusion step.
How long does a Milvus engagement take?
Four to eight weeks for a production deployment with ingestion, indexing, filtering and monitoring in place. Shorter if the data pipeline already exists. Ask for a scoping call.
Talk to us about your AI project
Tell us what you are trying to do, which systems are involved and what is in the way. We reply within one business day with an opinion and a suggested next step.