Банк вопросов из реальных собеседований

What is quantile regression and when is it useful to predict not the average, but, for example, the 90th quantile?

How to explain NDCG/recall and what online guardrails are needed for the ranking model?

The candidate talks about the RL project in drug discovery. How to explain the statement: state, action, environment, reward and quality metrics?

What does ROC AUC mean and why can it be understood as a ranking metric?

How to interpret ROC-AUC, what happens when scores are rounded and matched, and why does F1 use a harmonic mean of accuracy and completeness?

What variance reduction methods are applicable in product experiments and where does CUPED fit in?

What risks do coding assistants pose, and how do they fit into the team’s engineering process?

In the A/B dynamic delivery test, profit and margin can have heavy tail distributions. When is the t-test, bootstrap and z-test appropriate?

How to collect a dataset for a model that distinguishes the same car from a visually similar one?

For URL summarization, how would you extract useful content from a web page and remove navigation, ads, boilerplate and irrelevant DOM blocks?

After the first model, you need to understand what signs to leave and whether the model has become better. What offline metrics and checks should I use?

What does a typical business or product task look like? What do you refine when you have a top-level idea like raising a metric or automating a solution?

The model received some value of the average square error in the future period. How do you know if this is a good enough result?

There's a candidate generator and a ranker. What offline and online metrics to look for each stage?

The interviewer asks: what metrics were tracked and how did they understand that the implemented ML solution was really bringing benefits?

How are matrix equation, least squares, gradient descent and L1/L2 regularization related?

Two models have similar Precision@K and Recall@K, but one makes more money with more expensive, relevant products. How to adapt the offline metric?

What offline and product metrics should we use to evaluate a model that sends suspicious ads for moderation?

How would you measure quality for a model that restores punctuation and capitalization in ASR text?

What metrics to look for for a recommendation track system?

What offline and online metrics do a recommendation system need with visual and text features?

What metrics to use to evaluate rankings in recommendations or search?

How do you know if users like content on a news feed? What data to monitor and what biases can distort these metrics?

For a fraud-detection model, how would you choose validation metrics and a decision threshold when false positives and false negatives have different business costs?