Scenario

You are working on a language model that must handle long text and capture dependencies across distant tokens. The model uses transformer blocks instead of recurrence, and the core idea you need to explain is self-attention.

Question

Transformer architectures and self-attention mechanisms.