Implement Attention Component | Dataford Interview Questions

Implement a simplified version of a specific attention layer or model component from scratch. Implement scaled dot-product attention using pure Python lists, stable softmax computation, and an optional boolean attention mask. The function accepts query, key, and value matrices shaped [sequence_length, dimension], plus an optional mask shaped [query_length, key_length], and returns an output matrix shaped [query_length, value_dimension].

Constraints

1 <= len(query), len(key), len(value) <= 512
len(key) == len(value)
len(query[0]) == len(key[0])
All query, key, and value entries are finite numbers
If provided, mask has shape [len(query), len(key)] and contains booleans
Every query row has at least one unmasked key
The output has shape [len(query), len(value[0])]

Function Signature

def scaled_dot_product_attention(query, key, value, mask=None):

Your question is Implement Attention Component. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem

Constraints

1 <= len(query), len(key), len(value) <= 512
len(key) == len(value)
len(query[0]) == len(key[0])
All query, key, and value entries are finite numbers
If provided, mask has shape [len(query), len(key)] and contains booleans
Every query row has at least one unmasked key
The output has shape [len(query), len(value[0])]

Function Signature

def scaled_dot_product_attention(query, key, value, mask=None):

Your question is Implement Attention Component. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.