Toloka AI: ML System Design

System design for checking human/agent task outputs with LLM tools and evaluation metrics.

1 вопрос5 кейсов54 минДоступна аудиозапись

Аудио и материалы

0:00 / 53:38

Этап 4 из 4Toloka AIML / LLM Agent Engineer2025-09-11 - 2025-09-30

ML System Design в Toloka AI

The core design is not only an LLM prompt: it needs a structured error taxonomy, evidence gathering and evaluation against human reviewer outputs.
Metrics are set-valued and asymmetric: false accepts and false rejects have different operational costs.
Manual-review budget is a hard product constraint, not an afterthought.