Welcome to your interview.
The question is on your right: Explain Transformer Self-Attention. Take a moment with it first.
Talk your thinking through with me if you like - when you're confident, submit your answer and I'll grade it like a real screen (7/10 or better passes). Discussion and graded submissions share your five interviewer interactions, so spend them well.
You are working on a language model that must handle long text and capture dependencies across distant tokens. The model uses transformer blocks instead of recurrence, and the core idea you need to explain is self-attention.
Transformer architectures and self-attention mechanisms.