MHA, MQA, GQA and MLA
How do MQA, GQA, and MLA reduce attention during generation compared to conventional multi-headed attention?
Answer as you would in an interview
Answer aloud or write down a few key points. The text field is optional; the review opens after your attempt.