LTV-метрики, когда бизнесу нужна консервативная оценка
LTV-метрики, когда бизнесу нужна консервативная оценка
Ответить самому
Сначала сформулируйте ответ как на собеседовании, затем откройте разбор и оцените себя.
Короткий ответ
MSE optimizes point estimates, but acquisition decisions need downside risk. Add lower-bound or quantile metrics, calibration checks and business-threshold error analysis.
Полный разбор
MSE is useful for average point-prediction accuracy, but it treats overprediction and underprediction symmetrically. Marketing budget decisions are often asymmetric: overestimating LTV can make the company buy unprofitable traffic, while underestimating LTV may only leave some growth on the table.
A better evaluation includes risk-aware metrics. Train or evaluate lower quantiles with pinball loss, build prediction intervals, or report a conservative lower confidence bound for cohort LTV. Then check whether the lower bound is calibrated: for a claimed 10th percentile, roughly 10% of realized outcomes should fall below it.
Also evaluate decisions directly. If CAC is known, measure how often the model recommends buying traffic that later turns unprofitable, expected profit under the policy, and performance by channel/country/app segment. This connects model quality to the business action instead of only to a regression score.
Теория
When model output drives spend, uncertainty and asymmetric loss are part of the target, not just reporting extras.
Типичные ошибки
- Optimize only MSE and ignore overprediction risk.
- Report confidence intervals without calibration checks.
- Evaluate rows, but not the acquisition decision the model changes.
- Use one global threshold while acquisition economics differ by channel.
Как отвечать на собеседовании
- Mention quantile regression or lower confidence bounds.
- Tie the metric to CAC and expected profit.