Question
An ML engineer has trained a neural network by using stochastic gradient descent (SGD). The neural network performs poorly on the test set. The values for training loss and validation loss remain high and show an oscillating pattern. The values decrease for a few epochs and then increase for a few epochs before repeating the same cycle.
What should the ML engineer do to improve the training process?
- A. Introduce early stopping.
- B. Increase the size of the test set.
- C. Increase the learning rate.
- D. Decrease the learning rate. Correct
Explanation
The correct answer is D because decreasing the learning rate can help stabilize the training process and reduce the oscillating pattern observed in both training loss and validation loss, leading to potentially better performance on the test set according to AWS documentation.
