Банк вопросов из реальных собеседований

At which recommendation stage is recall more important, and where does precision become the priority?

How to explain Precision/Recall and what to check if the model is to be generalized to new regions or geographical features?

What is quantile regression and when is it useful to predict not the average, but, for example, the 90th quantile?

How to explain NDCG/recall and what online guardrails are needed for the ranking model?

The candidate talks about the RL project in drug discovery. How to explain the statement: state, action, environment, reward and quality metrics?

What does ROC-AUC measure, how is the ROC curve constructed, and why can AUC be interpreted as ranking quality?

What variance reduction methods are applicable in product experiments and where does CUPED fit in?

What risks do coding assistants pose, and how do they fit into the team’s engineering process?

Profit and margin can have heavy-tailed distributions. When are a t-test, bootstrap, or z-test appropriate in this A/B test?

How to collect a dataset for a model that distinguishes the same car from a visually similar one?

Before launching an A/B test, what should you define? What determines duration or sample size, and how do you interpret a p-value?

For URL summarization, how would you extract useful content from a web page and remove navigation, ads, boilerplate and irrelevant DOM blocks?

After the first model, you need to understand what signs to leave and whether the model has become better. What offline metrics and checks should I use?

What does a typical business or product task look like? What do you refine when you have a top-level idea like raising a metric or automating a solution?

The model received some value of the average square error in the future period. How do you know if this is a good enough result?

There's a candidate generator and a ranker. What offline and online metrics to look for each stage?

Which metrics did you track, and how did you determine that the deployed ML system actually helped the product?

Minimizing squared error corresponds to maximum likelihood under what noise distribution, and why?

Two models have similar Precision@K and Recall@K, but one makes more money with more expensive, relevant products. How to adapt the offline metric?

What offline and product metrics should we use to evaluate a model that sends suspicious ads for moderation?

A dataset has 95 ones and 5 zeros. The model always predicts one. Compute precision, recall, and ROC-AUC, then explain why the model is bad and how to choose a threshold.

How would you measure quality for a model that restores punctuation and capitalization in ASR text?

How to interpret ROC-AUC, what happens when scores are rounded and matched, and why does F1 use a harmonic mean of accuracy and completeness?

What metrics to look for for a recommendation track system?