ML System DesignHard
Running an audio event model on a user deviceThe model should run on a speaker or camera with a CPU and limitations on latency, privacy and power consumption. How do we design this solution?
Sber / GigaChat
Назад к тренажеруНайти понятиеOpen full bank
Банк вопросов из реальных собеседований
Направления, темы и вопросы из записей интервью. Фильтры ниже сохраняются в ссылке.
All questions
691 questions from real interviews
Topics
Difficulty
Company
Language
QuestionCompanySourceAction
ML System DesignHard
Device and data design for a medical computer-vision modelHow would you jointly design the device, recording protocol, and dataset for a computer-vision model on a medical endoscope?
Sber / GigaChat
ML System DesignHard
Separating the pricing action from the prediction targetWhy should the delivery fee or minimum order value not be the model target? What should the model predict if the system chooses the final offer?
Dodo
ML System DesignHard
What to do if invoice parsing is too expensive and slowPipeline for PDF invoices works, but processing has become slow and expensive. How to look for bottlenecks and optimize?
Vertex / BP
Metrics and A/BEasy
ROC-AUC: construction and interpretationWhat is ROC-AUC, how do you build the ROC curve, and what does the area mean probabilistically?
CIAN
Metrics and A/BEasy
Computing annual subscription LTV from a retention curveA monthly subscription costs 10 dollars. 100% of users pay month 1, then 50%, 40%, 30%, and after that each month is 10% lower than the previous one through month 12. Estimate annual LTV.
Adapty
Metrics and A/BEasy
Accuracy and completeness on the example of a diagnostic testHow to determine the accuracy and completeness of a binary classifier and how they are equal for a diagnostic test of a rare disease problem?
Revolut
Metrics and A/BMedium
A/B-tests of the recommendation modelHow to conduct offline and online experiments for a recommendation model? What is important in the A/B test: MDE, p-value, sampling, network effects and metrics?
GamerAM
Metrics and A/BMedium
Bootstrap for estimating LTV uncertaintyThere are historical users and calculated LTV. How to use bootstrap to estimate LTV variance and get a lower bound for traffic purchasing decisions?
Adapty
Metrics and A/BMedium
Bootstrap: what does it doWhat is bootstrap for and why doesn’t it by itself reduce the variance of the experiment?
MrBit
Metrics and A/BMedium
Debugging bad model metricsWhat to check if the ranking/model metric is unexpectedly low or the model looks overfit/underfit?
AgeCode
Metrics and A/BMedium
FP16 vs BF16What is the difference between FP16 and BF16 and why is BF16 often more stable for training?
Mirai
Metrics and A/BMedium
Imbalanced classification: metrics, loss and leakageHow to train and evaluate a model if a positive class is rare?
Waibee
Metrics and A/BMedium
Low-F1 classes in CV taxonomyThe model works well on average, but some classes have low F1. How to diagnose and repair?
2GIS
Metrics and A/BMedium
LTV/R365 and annual data driftIt is necessary to predict the revenue of the user for 365 days on early signs. Why can a direct approach work poorly?
EasyBrain
Metrics and A/BMedium
MAP/NDCG and the relationship to business effectHow to count MAP/NDCG for recommendations and why are these metrics not enough without a business link?
GRII RED
Metrics and A/BMedium
Metric learning for matching cars by photoHow to build a system that, based on photographs of advertisements, understands that there is another car in the report and removes erroneous matches?
Dubai Tech
Metrics and A/BMedium
Multi-output LTV curve forecastHow to use fresh incomplete cohorts if R365 is not yet known?
EasyBrain
Metrics and A/BMedium
NDCG, MAP and why ranking metrics are not directly optimizedHow does NDCG differ from MAP and why is it difficult to directly optimize such metrics by gradient descent?
HeadHunter
Metrics and A/BMedium
NDCG, MAP, BPR and WARP in RecommendationsWhat is the difference between NDCG/MAP and pairwise losses like BPR/WARP?
ZingBrain / Aviatrix
Metrics and A/BMedium
Offline and online metrics for recommendations and searchWhat metrics to look for when rolling out a new recommendation or search model?
Uzum
Metrics and A/BMedium
P-value and confidence intervalHow to jointly explain p-value, significance level and confidence interval?
MrBit
Metrics and A/BMedium
P-value and Bayes' formulaHow to briefly explain p-value, where it is used, and how to derive Bayes' formula through conditional probability and total probability?
Dubai Tech
Metrics and A/BMedium
BERT pooling, sentence transformers and metric-learning analogyHow can you get a sentence embedding from BERT, how do sentence transformers differ, and why is this similar to metric learning for image pairs?
X5