What Causes a Vector Database to Return Different Neighbors After Compaction?

A vector database can return different neighbors after compaction because the same embeddings may be reorganized into a newly built approximate-search structure.

For a local RAG server, the change often looks suspicious: no documents were intentionally re-embedded, yet a familiar query returns a slightly different top-k list after maintenance. The key distinction is between the vector values and the ANN index that searches them. Compaction can preserve the first while rebuilding the second.

Compaction Can Replace Several Search Segments With a New Index

A vector database often accumulates separate segments as documents are inserted, updated, and deleted. Compaction consolidates those pieces so the system has fewer structures to search and less obsolete data to carry.

Qdrant exposes optimizers that target the number and size of segments rather than treating the collection as one permanently fixed graph. When compaction creates a larger optimized segment, the physical search structure can be rebuilt even though the logical vectors are unchanged.

That distinction matters for a private RAG index: the embeddings can have identical numeric values before and after maintenance while the approximate search graph that connects them is different.

Approximate Nearest Neighbor Search Depends on Graph Topology

HNSW does not compare a query with every vector. It navigates a layered graph and follows a limited set of promising connections, so the route taken through the graph affects which candidates are examined.

Elasticsearch explains that segment merges may need to recompute HNSW graphs. A rebuilt graph can connect the same vectors differently because construction order, deletion state, and graph heuristics influence edges.

If two candidates have very similar distances, a small topology change can make one enter the candidate set while the other is never visited. The result is different approximate neighbors without any change to the embedding model.

Search Parameters Decide How Much of the New Graph Is Explored

After compaction, the database may search one larger graph instead of several smaller ones. The same top-k request can therefore traverse a different candidate landscape even when the configured search budget appears unchanged.

Weaviate documents the HNSW ef search-quality trade-off: a larger candidate list generally improves recall while increasing work. Near a ranking boundary, low search effort makes results more sensitive to graph construction.

A useful diagnostic is to compare approximate results with a high-ef or exact search on a small test set. If exact neighbors remain stable while ANN neighbors move, compaction changed the retrieval path rather than the vectors.

Featured

ZimaCube 2 Personal Cloud Home NAS

ZimaCube2

$799.00 - $2,499.00

Buy Now

Deletes and Updates Change Which Nodes Survive the Rebuild

Before compaction, deleted or replaced records may still exist physically with tombstones or segment-level bookkeeping. Searches filter them out, but their historical presence can influence the graph that was built earlier.

Milvus explains that HNSW stores an explicit graph structure in addition to raw vectors. Rebuilding after obsolete records are removed creates a graph from the surviving set.

That can alter local connectivity around a household document even when that document itself was never edited. A note may gain or lose a nearby bridge node, changing which region the ANN walk reaches first.

Ties and Near-Ties Can Flip Even When Distances Do Not

Many private corpora contain near-duplicates: repeated manuals, versioned files, photo captions, copied notes, or chunks with the same boilerplate. Their cosine or inner-product scores can be almost indistinguishable.

Pinecone’s HNSW explanation shows how graph navigation limits the vectors examined. When two items sit near the cutoff, a different candidate path or tie order can change the returned top-k without a meaningful semantic difference.

Applications should therefore avoid treating neighbor rank 7 versus rank 8 as a durable identity claim. Store stable document IDs and compare actual distances when deterministic behavior matters.

Exact Search Is the Boundary Between Data Drift and ANN Drift

The cleanest separation is to keep a small reproducible query set and record the embeddings, distance metric, exact top-k, approximate top-k, index settings, and database version before maintenance.

ZimaSpace’s discussion of embedding-domain changes in private retrieval covers a different failure class: the vector space itself changes. Compaction should be diagnosed separately because it can alter approximate retrieval while leaving that space intact.

ZimaSpace’s guide to document search and RAG workflows provides the application context: stable document identity and evaluation matter even when the ANN layer is allowed to be approximate.

If exact results change, inspect vectors, filters, normalization, metric, or data versions. If exact results stay fixed but ANN results move, the cause is index reconstruction, search effort, tie handling, or segment layout.

Compaction is therefore not expected to guarantee byte-for-byte neighbor order in an approximate index. Deterministic ranking requires stricter search or application-level tie rules.

FAQ

Does compaction change the embedding vectors?

Not by itself. A normal compaction or segment merge reorganizes storage and indexes. Embeddings change only if the application re-embeds, requantizes, renormalizes, or otherwise rewrites vector values.

Should exact nearest neighbors change after compaction?

They should remain the same when the surviving vectors, metric, and numeric representation are unchanged, aside from true score ties or floating-point implementation details.

Can rebuilding HNSW reproduce the exact old ranking?

Not always. HNSW is approximate and graph construction can be sensitive to insertion order, randomization, deletions, and implementation details. Exact ranking requires exhaustive or otherwise deterministic comparison.