Cross-Entropy Without Built-ins | Dataford Interview Questions

Given a matrix of token logits, implement a custom function to calculate cross-entropy loss for Mistral AI without relying on PyTorch's built-in loss function.

Use def cross_entropy_loss(logits, targets):, where logits is a non-empty list of equal-length rows shaped [batch_size][vocabulary_size], and targets contains one valid token index per row. Return the mean negative log-likelihood as a floating-point number. Use a numerically stable calculation so large logits do not overflow.

Constraints

1 <= len(logits) <= 1000
1 <= len(logits[i]) <= 100000
Every row has the same vocabulary size
len(targets) == len(logits)
0 <= targets[i] < len(logits[i])
Logits are finite real numbers

Function Signature

def cross_entropy_loss(logits, targets):

Your question is Cross-Entropy Without Built-ins. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.

Problem

Given a matrix of token logits, implement a custom function to calculate cross-entropy loss for Mistral AI without relying on PyTorch's built-in loss function.

Constraints

1 <= len(logits) <= 1000
1 <= len(logits[i]) <= 100000
Every row has the same vocabulary size
len(targets) == len(logits)
0 <= targets[i] < len(logits[i])
Logits are finite real numbers

Function Signature

def cross_entropy_loss(logits, targets):

Your question is Cross-Entropy Without Built-ins. Start with the requirements in the Question tab.

Run and submit as often as you like. When you're ready, talk me through your approach or go straight to the code.