Position information in Transformer
Why does the attention mechanism itself not encode the order of tokens, and what absolute and relative positional schemes are used in language models?
Answer as you would in an interview
Answer aloud or write down a few key points. The text field is optional; the review opens after your attempt.