Hello and welcome to this video on overfitting. We will start this video with a summary of supervised learning, then we will define what overfitting is, and we will list a few common mistakes that lead to overfitting. Let us recap what we have learned so far in supervised learning. We have seen two supervised learning techniques: Linear regression and logistic regression. Linear regression deals with problems where the target variable takes continuous values on the real axis. While logistic regression deals with problems where the target variable is binary and can only be zero or one. Linear regression is defined with the linear hypothesis that connects the features x to the outcome y. The logistic regression connects the features to the probability of the outcome to a non-linear function called sigmoid. 

The net effect of the sigmoid is to map all the possible values of b plus X dot w into the interval zero to one. Finally, we have defined a cost for each of them using the mean squared error for the linear regression and the cross entropy or log loss for the logistic regression. Notice that here we have introduced a vector notation where X dot w is equal to the sum of products of the components of x and w to extend the model to data sets with multiple features. All these means is that now w is a vector of size M, where M is the number of features, while X is a matrix of size N by M. Where N is the number of records in our data sets and M is the number of features. For the case of only one feature, the sum only has one term and we're back to the case we previously discussed. We have also learned to split our data in two parts, a training set and a test set. Now, let's talk about one thing to watch out for, overfitting. Overfitting happens when our model learns the probability distribution of the training set too well. And, it's not able to generalize to the test set with the same performance. Think of this as learning things by hard without really understanding them. In a new situation, you would probably be lost and probably underperform. A very simple way to check for overfitting is to compare the cost and the performance metrics on the training and the test set. If our training score is much better than the test score, we can be sure that we are overfitting. For example, let's say we're performing a classification and we measure the accuracy to be 99%. If this is the score on the training set and on the test set, we only obtain 85%, the performance of our model is worse on the test set than it is on the train set, and therefore, we are overfitting. How can we avoid overfitting? There are several actions we can take. 

The first simple check is to make sure there are train test split is performed correctly and both the train and test sets are representative of the whole population of features and labels. Also, make sure that you're sampling the training and test set randomly, so that the order of the data doesn't affect your sampling. Third, make sure not to choose too small at test set, which could simply not contain enough data to be a good test. And finally, don't choose too small at training set, which could cause your model not to learn generally enough rules. If the train test split seems correct, it could be the case that our model as simply too much freedom and therefore, learns by hard the training set. This is usually the case if the number of parameters in the model is much greater than the number of data points. And, it's definitely a very common problem in neural networks. In order to mitigate this, we can either reduce the complexity over the model or use a technique called regularization, that we will learn later in this course. So, in this video, we've reviewed the main ingredients of supervised learning and these are data, labels, hypothesis and cost. We have also discovered that overfitting corresponds to inability of our model to generalize to previously unseen data. Finally, we have described a few common mistakes to avoid. Thank you for watching and see you in the next video.