LLM memory math and quantization for a 72B model
A 72B-parameter LLM is served on an A100 80GB. Estimate whether FP16 fits and explain what quantization changes.
Answer as you would in an interview
Answer aloud or write down a few key points. The text field is optional; the review opens after your attempt.