Weights in CNNs
The course is part of this learning path
Convolutional Neural Networks
In this course, discover convolutions and the convolutional neural networks involved in Data and Machine Learning. Introducing the concept of tensor, which is essential for everything that follows.
Learn to apply the right kind of data such as images. Images store their information in pixels, but you will discover that it is not the value of each pixel that matters.
- Understand how convolutional neural networks are essential to the fundamentals of Data and Machine Learning.
- It is recommended to complete the Introduction to Data and Machine Learning course before starting.
Hello, and welcome to this video on weights in convolutional neural networks. In this video, we will count the weights in a fully-connected network and compare it with the weight count in a convolutional neural network. One thing we've only briefly mentioned is how the convolutional filters are chosen. The filters in the convolutional layers are not predefined. The values in their tensors are the weights that the network has to learn in order to solve the classification problem. In other words, when we train a convolutional network, we are forcing it to learn the perfect patterns to solve the problem. Let's compare the parameter count for a fully connected layer and a convolutional layer. If the input image has a size of 10 by 10 we have 100 input numbers. If we use a fully connected layer with 32 nodes, each of them is connected to the 100 inputs with 100 independent weights. This gives us for the first layer 3200 weights. In the convolutional case, we still have 100 input numbers arranged as pixels in the image of size 10 by 10.
If the first layer has 32 convolutional filters with a filter size of 3 by 3 pixels we have nine weights for each filter. This gives us only 288 weights to train, which is much less than before. It's more than one order of magnitude less. Besides, since our weights are arranged in a 2D shape there is a better chance to capture local information from both rows and columns. So in a convolutional layer we have less parameters because we are sharing the weights with many pixels in the image through the convolution. In this way we are trading parameters for some additional computation. In this video we've seen that the number of weights in a CNN layer is much smaller than the number of weights in a fully connected. Also, the architecture of the CNN is better at capturing local parameters. Thank you for watching, and see you in the next video.
About the Author
I am a Data Science consultant and trainer. With Catalit I help companies acquire skills and knowledge in data science and harness machine learning and deep learning to reach their goals. With Data Weekends I train people in machine learning, deep learning and big data analytics. I served as lead instructor in Data Science at General Assembly and The Data Incubator and I was Chief Data Officer and co-founder at Spire, a Y-Combinator-backed startup that invented the first consumer wearable device capable of continuously tracking respiration and activity. I earned a joint PhD in biophysics at University of Padua and Université de Paris VI and graduated from Singularity University summer program of 2011.