Calculation of KV-cache memory
Why can keys and values of past tokens be reused in step-by-step generation and on what multipliers depends the volume of the KV cache?
Answer as you would in an interview
Answer aloud or write down a few key points. The text field is optional; the review opens after your attempt.