You are building a regression model to predict the wind speed at any point on earth during any time of the day. You are training your model with Gradient Descent and you notice that every time you run a training, you get different final loss values. Some models are great, some average, and some very poor.
How can you fix this problem?
Choose the correct answer from the options below.
I'm sorry, your answer is not correct.
There are many reasons why you might see this behavior, but in this scenario we assume that you're dealing with a very irregular loss surface with lots of false minima. With a step size that is set too small, Gradient descent keeps getting stuck in a false minima and seldom finds the true optimal solution, so you're seeing different loss values after every training run.