Метрики классификации, ties в ROC-AUC и F1
Метрики классификации, ties в ROC-AUC и F1
Ответить самому
Сначала сформулируйте ответ как на собеседовании, затем откройте разбор и оцените себя.
Короткий ответ
ROC-AUC is the probability a random positive is scored above a random negative. Score ties contribute half credit. F1 is the harmonic mean, so a low precision or recall strongly limits the final value.
Полный разбор
ROC-AUC evaluates ranking, not a fixed threshold. It can be interpreted as the share of positive-negative pairs where the positive object receives a higher score than the negative object, with ties usually contributing 0.5.
If scores are rounded, ordering can be lost. Pairs that were correctly ordered may become ties, which lowers their contribution from 1 to 0.5. Pairs that were incorrectly ordered may also become ties, improving from 0 to 0.5. In practice rounding often reduces resolution and can change AUC even if classification at one threshold looks unchanged.
F1 is 2 * precision * recall / (precision + recall). The harmonic mean is closer to the smaller value, so a model cannot compensate terrible recall with excellent precision, or the reverse. Use F-beta when one side is more important.
Теория
Ranking metrics and threshold metrics answer different questions; score ties are part of the ranking definition.
Типичные ошибки
- Say ROC-AUC is accuracy over thresholds.
- Ignore the 0.5 contribution of tied scores.
- Treat F1 as an arithmetic average.
- Use F1 when the business has asymmetric error costs but no beta choice.
Как отвечать на собеседовании
- Give the pairwise interpretation of ROC-AUC.
- For F1, say explicitly that harmonic mean penalizes the smaller metric.