What are the trade-offs of using different optimizer variants, such as AdamW versus SGD with momentum, when training large-scale models?
