Diagnosing Vanishing and Exploding Gradients

Medium

Neural NetworksDeep Learningoptimization

Scenario

You are training a deep neural network and notice unstable learning. Loss may plateau early, become NaN, or improve only in shallow layers.

How do you diagnose and mitigate vanishing or exploding gradients during the training of deep neural networks?

You are training a deep neural network and notice unstable learning. Loss may plateau early, become NaN, or improve only in shallow layers.

How do you diagnose and mitigate vanishing or exploding gradients during the training of deep neural networks?

Practice Python

Python 3.10

Open on desktop for the full Python editor with syntax highlighting and autocomplete.

Up next

All questions/Machine Learning

Asked 2mo ago|

Lucid Motors

Medium

Neural NetworksDeep Learningoptimization

You are training a deep neural network and notice unstable learning. Loss may plateau early, become NaN, or improve only in shallow layers.

How do you diagnose and mitigate vanishing or exploding gradients during the training of deep neural networks?

You are training a deep neural network and notice unstable learning. Loss may plateau early, become NaN, or improve only in shallow layers.

How do you diagnose and mitigate vanishing or exploding gradients during the training of deep neural networks?

Practice Python

Python 3.10

Open on desktop for the full Python editor with syntax highlighting and autocomplete.

Up next

Diagnosing Vanishing and Exploding Gradients | Dataford Interview Questions