After hybrid retrieval, you can give several articles to LLM. When is this justified, and when is it better to leave a regular reranker and a list of results?
Банк вопросов из реальных собеседований
Направления, темы и вопросы из записей интервью. Фильтры ниже сохраняются в ссылке.
ML System Design
274 questions from real interviews
What labels should we collect to teach how to select a test question and how to deal with the fact that we only see answers to the questions shown?
What features of existing models, maps, conditions and source quality should I add after vector search?
Sketch the online architecture for query parsing, candidate generation, ranking and blending. How do services communicate and fail safely?
A query like "book" returns a million relevant products. How not to waste the whole million with a heavy model?
What model files, metrics, and signals are stored after launch to replicate a decision and notice a change in the format of a particular bank?
You need to get high-quality embedding sentences for semantic search. What data, positive and negative pairs and loss functions to use?
If the new carousel was not yet in the product, how to assess the relevance of its recommendations before the experiment and not take the absence of the old display as a negative label?
How do you determine the dataset string, time partition, and metrics of the price model or recommendations so that the offline valuation reflects the actual application?
Which offline and online metrics would you use for a similar-items recommender, and what pitfalls are easy to miss?
How to arrange a platform where the main search for candidates is considered a batch, and the operational layer changes the order according to the fresh actions of the user?
How to get training pairs of queries and relevant road segments with a limited manual markup budget?
How to improve the order of the found road segments with a more accurate model and notice the degradation of the search in time?
The bank receives PDF statements from other banks. How to turn documents into verifiable compliance signals?
Transactions and the state of the exchange glass are given. How do you define a target, build signs, do a temporary check, and choose the first basic solution?
In a large archive of trips, you need to find rare traffic situations by text request. How to determine the user, search unit, metrics and the first version of the system?
What features of a user, product, cart and their interactions should be submitted to an MLP or booster to rank recommendations?
How should the production system process finished calls, write bookings and avoid operational issues such as double-booking a slot?
After the basic option and the ranker, you need to design the operation: where to look for candidates, where to store features and how to update recommendations when changing the basket?
In production RAG there is FastAPI, vector DB, ranker service, MLflow, Docker and self-hosted LLM. How to describe the request path and service areas of responsibility?
How to select a A/B dynamic delivery test partition unit, evaluate the MDE and test the experiment before starting?
In the call center, you need to select a security question to verify the client: fairly secure, but not too complicated. How to build a ML question ranking system?
Why can keys and values of past tokens be reused in step-by-step generation and on what multipliers depends the volume of the KV cache?
Большая temporal model хорошо ловит события, но слишком дорогая по latency и compute. Как сжать ее для production?