Построение и обновление HNSW/Qdrant пайплайна векторного поиска
Построение и обновление HNSW/Qdrant пайплайна векторного поиска
Ответить самому
Сначала сформулируйте ответ как на собеседовании, затем откройте разбор и оцените себя.
Короткий ответ
Choose ANN parameters by recall/latency/memory, define how embeddings are written and refreshed, and monitor freshness, index build success, candidate quality and serving/query cost.
Полный разбор
Moving from brute-force search to HNSW changes both quality and operations. Build an exact-search sample, then compare ANN recall@K, p95 query latency, memory, build time and downstream recommendation metrics. Tune HNSW parameters such as M, efConstruction and efSearch for the target recall/latency point.
The pipeline needs an artifact contract: compute embeddings, validate counts and dimensions, upsert into Qdrant with stable ids and payload metadata, build or refresh the index, then mark the version ready. For batch systems, decide whether full rebuild every few days is acceptable or whether incremental upserts are needed for freshness.
Operational checks should cover missing embeddings, stale index versions, failed upserts, schema changes, vector normalization, payload filters and rollback to the previous index.
Теория
ANN is not just a faster library call; it is a quality-latency-freshness tradeoff that needs a publish contract.
Типичные ошибки
- Compare ANN methods without exact-search ground truth.
- Forget vector normalization and metric compatibility.
- Overwrite the live index without a rollback path.
- Ignore payload filters and metadata needed by downstream ranking.
Как отвечать на собеседовании
- Mention an exact-search evaluation sample.
- Talk about full rebuild versus incremental upsert.