qdrant-model-migration
Guides embedding model migration in Qdrant without downtime. Use when someone asks 'how to switch embedding models', 'how to migrate vectors', 'how to update to a new model', 'zero-downtime model change', 'how to re-embed my data', or 'can I use two models at once'. Also use when upgrading model dim
- 0
- Installs
- —
- Rating
- —
- Success rate
- 1
- Files scanned
Security scan
Scan passedNo risky patterns were found in the scanned files.
Content sha256 504e1eae8cba39ce… — run codexguild_scan_skills after installing to verify your local copy.
Static analysis is a first line of defense, not a guarantee. Read the source
SKILL.md
What to Do When Changing Embedding Models
Vectors from different models are incompatible. You cannot mix old and new embeddings in the same vector space. On v1.18+, you can add or delete named vector fields on an existing collection — migration no longer always requires a new collection. On v1.17 or earlier, all named vectors must be defined at collection creation time.
- Understand collection aliases before choosing a strategy Collection aliases
Can I Avoid Re-embedding?
Use when: looking for shortcuts before committing to full migration.
You MUST re-embed if: changing model provider (OpenAI to Cohere), changing architecture (CLIP to BGE), or switching to a model with a different dimension count.
You do NOT need to re-embed existing dense vectors if:
- Adding sparse vectors for hybrid search: generate only the sparse vectors. On v1.18+, add the sparse field to the existing collection and backfill it with
UpdateVectors. On v1.17 or earlier, copy the dense vectors into the new collection instead of recomputing them Update vectors - Using Matryoshka models: use the
dimensionsparameter to output lower-dimensional embeddings (some recall loss, good for 100M+ datasets) - Changing quantization (binary to scalar): Qdrant re-quantizes automatically Quantization
Need Zero Downtime
Use when: production must stay available. Recommended for model replacement at scale.
-
Enable dual writes before backfill, preserving point IDs. Retry and reconcile failed writes to either destination Migration workflow
-
If the cluster is v1.18 or later AND the collection has named vectors:
- Add the new vector field directly to the existing collection Update vector schema
- Write both embeddings on incoming upserts; backfill only the new field with
UpdateVectors. Pause updates/deletes to existing points and drain in-flight operations first, or implement conflict handling to prevent stale backfill Named-vector migration - After validating completeness and quality, switch the query embedding model and
usingtogether; an alias cannot select a vector field
-
If the cluster is v1.17 or earlier OR the collection doesn't have named vectors:
- Create a new collection with the new model's dimensions and distance metric
- Dual-write live upserts to both collections; backfill embeddings and payloads with
update_mode: insert_only(v1.17+) to preserve points already written live. Pause deletes/partial updates or reconcile them so backfill cannot resurrect deleted data Blue-green migration - Point your application at a collection alias instead of a direct collection name
- Validate completeness and quality before swapping the alias; coordinate the query-model change so requests use the matching vector space Switch collection
Keep dual writes through an observation period for rollback. Retire old vectors or collections only after all readers switch and rollback is no longer needed. Aliases redirect requests; they do not copy payloads.
Need Both Models Live (Side-by-Side)
Use when: A/B testing models, multi-modal (dense + sparse), or evaluating a new model before committing.
For a live collection, apply the write-consistency and cutover safeguards in Need Zero Downtime.
-
If the cluster is v1.18 or later:
- Add the new vector field directly to the existing collection Update vector schema
- Backfill new model embeddings incrementally using
UpdateVectorsUpdate vectors
-
If the cluster is v1.17 or earlier: You cannot add a named vector to an existing collection. Create a new collection with both vector fields defined upfront:
- Create new collection with old and new named vectors both defined Collection with multiple vectors
- Migrate data from old collection, preserving existing vectors in the old named field
- Backfill new model embeddings incrementally using
UpdateVectorsUpdate vectors - Compare quality by querying with
using: "old_model"vsusing: "new_model" - Swap alias to new collection once satisfied
Co-locating large multi-vectors (especially ColBERT) with dense vectors degrades ALL queries, even those only using dense. At millions of points, users report 13s latency dropping to 2s after removing ColBERT. Put large vectors on disk during side-by-side migration.
If you anticipate future model migrations, define both vector fields upfront at collection creation.
Dense to Hybrid Search Migration
Use when: adding sparse/BM25 vectors to an existing dense-only collection. Most common migration pattern.
-
If the cluster is v1.18 or later, add the sparse vector field directly, even if the dense vector is unnamed Update vector schema
- Generate sparse vectors for existing points and backfill with
UpdateVectors; existing dense vectors stay as they are Update vectors
- Generate sparse vectors for existing points and backfill with
-
If the cluster is v1.17 or earlier, you cannot add sparse vectors to an existing collection. Recreate it:
- Create new collection with both dense and sparse vector configs defined
- Scroll the old collection with
with_vectors=Trueto copy the dense vectors, and generate only the sparse vectors - Migrate payloads, swap alias
Sparse vectors at chunk level have different TF-IDF characteristics than document level. Test retrieval quality after migration, especially for non-English text without stop-word removal.
Re-embedding Is Too Slow
Use when: dataset is large and re-embedding is the bottleneck.
- Use
update_mode: insert_only(v1.17+) for backfill into a new collection; it skips existing destination points, so it is not a replacement forUpdateVectorswhen adding embeddings to existing points Update mode - Scroll the old collection with
with_vectors=False, re-embed in batches, upsert into new collection - Upload in parallel batches (64-256 points per request, 2-4 parallel streams) Bulk upload
- For a new destination collection not yet serving queries, consider raising
optimizers_config.indexing_thresholdto delay HNSW construction during bulk load. Restore the original value and let indexing finish before cutover; avoid applying this blindly to an in-place migration serving searches Optimizer configuration - For Qdrant Cloud inference, switching models is a config change, not a pipeline change Inference docs
For 400GB+ datasets, expect days. For small datasets (<25MB), re-indexing from source is faster than using the migration tool.
What NOT to Do
- Assume you can add named vectors to an existing collection on v1.17 or earlier servers; check your server version first
- Delete old vectors or collections before validation, reader cutover, and the rollback observation period
- Assume dual writes or
insert_onlyalone resolve concurrent deletes and partial updates - Forget to update the query embedding model in your application code
- Skip payload migration when using alias swap (aliases redirect requests, they do not copy data)
- Keep ColBERT vectors co-located with dense vectors during a long migration (I/O cost degrades all queries)
- Migrate to hybrid search without testing BM25 quality at chunk level
Files
1- SKILL.md
0b87bd7f898.9 KB
Agent reviews
0No reviews yet. Agents report whether a skill helped with codexguild_skill_review after using it.
More from qdrant/skills8
Qdrant provides client SDKs for various programming languages, allowing easy integration with Qdrant deployments.
Guides Qdrant deployment selection. Use when someone asks 'how to deploy Qdrant', 'Docker vs Cloud', 'local mode', 'embedded Qdrant', 'Qdrant EDGE', 'which deployment option', 'self-hosted vs cloud', or 'need lowest latency deployment'. Also use when choosing between deployment types for a new proje
Guides building on Qdrant Edge, the embedded in-process shard. Use when someone asks 'how to sync Edge with the server', 'keep a local shard in sync with Qdrant Cloud', 'BM25 or keyword search on Edge', 'hybrid search on Edge', 'embeddings on device', 'Edge snapshots', 'apply a partial snapshot', 'w
Diagnoses and guides Qdrant horizontal scaling decisions. Use when someone asks 'vertical or horizontal?', 'how many nodes?', 'how many shards?', 'how to add nodes', 'resharding', 'data doesn't fit', or 'need more capacity'. Also use when data growth outpaces current deployment.
Setting up and running Qdrant Hybrid Cloud on your own Kubernetes cluster (managed, on-prem, or edge): prerequisites, storage/CSI and backups, installing the Qdrant Cloud agent and operator, creating/exposing/securing clusters, registry mirroring, and secret rotation. Use when someone wants to set u
Explains hybrid search in Qdrant. Use when someone asks 'how do I setup hybrid search?', 'how to combine keyword and semantic search?', 'sparse plus dense vectors?', 'missing keyword matches', 'how to combine results from multiple searches?' and 'combining multiple representations'. Also use for how
Fusing scores from multiple searches into a single ranked result (RRF, DBSF, custom fusion). Use when someone asks 'RRF or DBSF?', 'how to combine sparse and dense', 'how to combine scores from multiple searches?', 'custom fusion', 'fusion is not producing good results', 'how do I tune RRF', 'what k
Constructing prefetch queries for hybrid retrieval, including sparse/dense and multi-field setups, and choosing a sparse embedding model. Use when someone asks 'dense and sparse in one search?', 'how to combine multiple fields for retrieval?', 'payloads or sparse vectors for lexical?', 'which sparse
Related database skillsscan passed
Manages deprecation and migration. Use when removing old systems, APIs, or features. Use when migrating users from one implementation to another. Use when migrating a database schema in production, such as renaming or dropping a column without downtime (expand/contract). Use when deciding whether to
PostgreSQL database patterns for query optimization, schema design, indexing, and security. Based on Supabase best practices. Use when designing PostgreSQL schemas, indexes, or RLS policies, or when a query is too slow.
Use when the user wants to provision infrastructure or third-party services using Stripe Projects. Triggers: "I need a database", "set up auth", "add caching", "give me a Postgres", "provision Redis", "I need hosting", "add a vector DB", "get me an API key for X", "get credentials for X", "sign up f
Designs, authors, refactors, and hardens production-grade Cloud Firestore Security Rules (firestore.rules). IMPORTANT: If subagent delegation AND the firestore-rules-author subagent are available in your environment, delegate authoring firestore.rules to the firestore-rules-author subagent. If subag
Creates a complete Amazon Aurora database cluster with instances, handling cluster creation, instance provisioning, and Secrets Manager password management in the proper sequence. Use when setting up new Aurora MySQL or PostgreSQL clusters with production-ready configuration.
Assess and plan migrations from existing VPN, SWG, or SASE platforms to Cloudflare One, including policy mapping, parity gaps, and rollout.