What will the runtime pipeline look like for a new procurement application and what to do with new customers, new suppliers and rare categories?
Банк вопросов из реальных собеседований
Направления, темы и вопросы из записей интервью. Фильтры ниже сохраняются в ссылке.
ML System Design
274 questions from real interviews
If embeddings, scores or recommendation lists are considered offline and lie in the S3/DWH, how can these results be safely transmitted backend/serving?
How does a visual encoder turn an image into tokens and how do they connect to a language model along with text?
The model is already able to predict the probability of return. How to use it in the product and where to store the signs?
Explain how LLM tool/function calling works end to end: tool schema in the prompt, model output, real tool execution and final user response.
The team wants to add new features or models to the ranking service. How do you do that safely?
For the article you need to show short suggest questions or tips. How to get them from the text of an article without degrading the quality of the search?
How to compare forecast models if LLM-extractor can know future facts from pretraining?
How was quality assessed: how well are you able to conduct a dialogue, answer a question, or search for the necessary documents?
How do you know if the article search or RAG system is working well? Which offline and online metrics should I use?
In a casino product, the sales team needs to understand as early as possible whether the new player will become a VIP in terms of deposits and turnover. How to formulate a ML-task, target, forecast horizon and business action?
How to technically build a model that determines the event by audio: the barking of a dog, the sound of a door, broken glass and similar classes?
We need to build a system where the advertiser looks at the CTR of campaigns. Given 200 billion impressions per day and a CTR of about 1%. How to start system design with numbers?
The product has a database of articles. The user can see hints or ask a free question. How to separate these two modes in search design?
The moderation model requires classes and data. How to collect labels, handle imbalances, and not mix different policies into one noisy dataset?
Fairmarkit is a marketplace for corporate procurement: the customer creates a request, and the system suggests suitable suppliers. How to formulate the ML-problem of supplier selection before choosing a model?
We need to deploy a text moderation service on BERT/DistilBERT. How to design input/output, policy layer, thresholds and routing actions?
If a user has added a ring, should they recommend more rings? How to formulate goals and limits for recommendations in the basket?
You mentioned seasonality. How to work with it in features for recommendation systems, forecasts or product analytics?
The customer sees the shipping cost or free shipping threshold in the cart. The catalog and basket change, but you can not unexpectedly show a different price on the checkout. How do you set the line between accuracy, delay and cost?
What does the language model get at the input, what returns at one step, and how does the sequence of response come from it?
What embedding architecture did you build for RAG: a regular retrieval pipeline or something more complex?
The moderation model works in production. What metrics should you look at offline, online and after launch in order to control the quality and load on manual checking?
After basic latency questions, the interviewer asks: what other anomalies can be noticed in the market-data file?