You are adapting a pretrained language model to perform well on text from a specialized domain rather than general internet text. The data uses domain vocabulary, formatting, and label definitions that differ from what the base model saw during pretraining.
How would you fine-tune a language model for a domain-specific task?
Choosing a pretrained transformer for a domain taskDesigning a realistic preprocessing pipeline for domain textFine-tuning strategy for supervised classificationEvaluating with F1 under class imbalance