Hello, and welcome to this video on the learning rate. In this video, we will talk about the learning rate. When we perform our update step, we update the value of w to w minus the derivative of the cost. Doing this, we move by a quantity determined by the rate of change in the cost with respect to the weight, w. This could be a problem in two ways. If the cost function is very flat, we will move very, very slowly towards the minimum, and vice versa, if the function is very steep, we might end up jumping beyond the minimum. The solution to both problems is to introduce a tuneable knob that allows us to decide how big of a step to take in the direction of the gradient. This is called learning rate, and it is usually indicated with the letter alpha. If we choose a small learning rate, we will move by tiny steps.

 If we choose a large learning rate, we will move by large steps. However, we must be very careful, because if the learning rate is too large, we will actually run away from the solution. At each new step, we move towards the direction of the minimum, but since this step is too large, we overshoot and go beyond the minimum, at which point, we reverse course and repeat, going further and further away. In this video, we've introduced the tuneable parameter called learning rate, and we've learnt that it's necessary to tune it in order for our algorithm to optimally converge to the minimum cost. Thank you for watching and see you in the next video.