Autoregression generation and cache of keys and values
How does the model generate text token by token and what work saves the cache of keys and attention values?
Answer as you would in an interview
Answer aloud or write down a few key points. The text field is optional; the review opens after your attempt.