QuestionMediumML System DesignProduction ML question from a screening call · Unconfirmed Dubai LLM role
Serving open-source LLMs on a multi-GPU server
You have a multi-GPU server and want to host one or more open-source LLMs. What software stack and design choices would you use?
Answer as you would in an interview
Answer aloud or write down a few key points. The text field is optional; the review opens after your attempt.