This is the last section of Chapter 6. In this section, we're going to talk about product recommendation. First, we introduced the basic concept and application of the recommendation system, focusing on the recommendation of product on the e-commerce platform. Then, we introduced a product recommendation experiment based on collaborative filtering algorithm and conduct the actual operation demonstration of the experiment.

Imagine the day in your daily life when you have breakfast in the morning, you turn on the music app on your mobile phone and the app recommends your favorite songs for you, which starts your day in a good mood. After breakfast, you're sitting on the subway on your way to work and you open the shopping app. You want to choose a coat in season for yourself. On the recommendation page, you can see the colors and styles you like as well as the brand of clothing that suits your dressing style.

When you want to order a take out for lunch, the takeout app will recommend the food you like based on taste, distance, and price. After a whole day of working when you get home and want to relax, whether you want to flick through some short videos of reading ebook, your mobile app can always find out which one you want in a sea of information. In the era of internet, we enjoy the convenience brought by the recommendation system all the time. Through the recommendation system, we can easily get what we want from the mass of information. It's like your old friend who can always give you reasonable suggestions when you're faced with a large number of options and difficult to choose.

Now, let's make a general introduction to the background, concepts, and characteristics of recommendation system. Recommendation system is not born out of thin air. And this birth is closely related to the development of the internet and information technology. With the rapid development of information technology, the amount of information produced by human beings is increasing geometrically, which brings a new problem, information overload. To put it simply, information overload means that in the era of information, because we are exposed to too much information everyday is far beyond our ability to accept and process them which leads to our inability to filter and outlines information effectively.

The vast amount of information increases a lot of uncertainty and also increases the anxiety inside people. In the time before internet and TV media is developed, for example, we can only get news through the newspaper. And now when we open our mobile phone, anytime and anywhere, we are receiving news from all over the world and we are always feeling fed up with them. In this way, we get our horizons broadened, but our attention has also been in road by the vast amount of information. It reduced the efficiency of information retrieval. In the meantime, we became confused and disoriented.

In the time without online shopping, if we want to buy a toothbrush, we can easily go to the store near our home to buy it. However, when we shop online, we tend to have a fear of choice in the face of a large number of similar products and sellers also worry about whether their products can be browsed by those who really need them. The biggest problem we face is not we can't learn, we don't know, but we can't decide, we can't make choices. 

At the same time, the way of information storage is also changing. With the development of social networks and video platforms, internet information is gradually transitioning from structured data, such as taxed to unstructured data, such as pictures, audio, and video, which greatly increases the difficulty of information retrieval. These are normal things in the time of information overload. So, an effective information filtering mechanism is particularly important today. Therefore, the recommendation system came into being.

The so-called recommendation system is an information filtering application which is used to recommend information or objects that users may like, whether it is products, news, or people with the same interest as you. Recommendation system has the following two characteristics, initiative and personalized. The initiative of the recommendation system is shown in that it does not need users to provide explicit requirements, but by analyzing the data generated by users and mining their behavior habits, it actively recommends information that users may be interested in.

Personalization means that the recommendation system can break through the long tail effect or information and meet the personalized preference of users to the greatest extent. In short, the long tail information is information that is less popular, such as a very low mainstream album that cannot be easily found in the traditional ranking search method, but can be recommended if it fits the user's personal preferences. The picture shows the main applications of recommendation system on the internet.

The first is the e-commerce platform that we are most familiar with. The platform with mild preferences for certain products according to our daily browsing and purchasing records and recommend the products we may need. For sellers, their products can be browsed by people who really need them. The second is social networks such as Facebook, Twitter, Weibo and so on. On social network platforms, we'd like angry tweets, our favorite messages, and follow and interact with other users. Recommendation systems can help us more if they reach the information and users we want to follow.

Then there are online music and video platforms. Recommendation systems help us to find the styles and genres we like more quickly while also making music and video creator's audiences more engaged. Finally, there's internet advertising. Compared with the traditional way of large scale advertising, the accurate advertising through the recommendation system is a convenient and cost saving way for advertisers. To sum up, the recommendation system solves two problems, one is to enable consumers to find the information they want from a large amounts of information. The other is to enable the information produced by information producers, to stand out from the large amount of information and get users attention.

It can be said that the recommendation system is a bridge connecting users and information and every participant in the internet ecosystem can get convenience from it. Now, we will take our most familiar e-commerce platform as an example to introduce the model of product recommendation. Amazon is one of the earliest e-commerce platforms in the world to use the recommendation system. And the use of the recommendation system has also brought huge profits to the platform. What is shown in the picture is the daily deals and promotions interface of Amazon platform, which recommends discounts and upcoming products to users. This is a kind of routine recommendation.

Next, we will introduce two kinds of recommendation methods of e-commerce platform, routine recommendation, and personalized recommendation. Routine recommendation refers to top sellers chooses some fixed products to be placed in the recommendation place, which does not vary according to different users and it's mainly applied in activities and promotions. The rules for the configuration of products are often fixed such as the list of sales and collections.

As we often see with top sellers, personalized recommendation refers to the collection of user information and the establishment of users portraits and the use of appropriate recommendation algorithms and methods to recommend personalized products for users. The click, browse, and purchase behavior of users recommendation results can also be used as a reference for optimizing the recommendation system. If I am an internet practitioner and interested in electronic products and I often browse the information of electronic products, then the e-commerce platform will be inclined to recommend electronic devices, such as notebook and earphone from me.

E-commerce platforms often combined routine recommendation with personalized recommendation to achieve the overall optional recommendation effect. The use of fragmentation system plays a huge role in the development of e-commerce platforms. First, it can turn site visitors into buyers or potential buyers. Even if the user is just idly browsing a shopping site, the system's precise recommendations based on the user's browsing preferences tend to arouse the user's desire to make a purchase. Even if they don't plan to make a purchase at the moment, they are more likely to add it to the shopping cart. Secondly, it can improve the cross selling ability of the shopping website and the transaction conversion rate.

Cross selling refers to the discovery of various needs of existing consumers and so various related services and products to customers. After we purchase a product, the system will often recommend relevant products for us. For example, after we purchase a mobile phone, we may receive a recommendation of the mobile phone cover. The transaction conversion rate is the radio of the volume of a product to the number of page views. With the help of the recommendation system, we can reduce aimless browsing, directly find and buy the goods we need and the transaction conversion rate will naturally increase.

Finally, the recommendation system can improve customer's loyalty to the platform. If we can always easily find the products we want the most on an e-commerce platform, then we will get used to shopping on this platform. For the platform, it will improve users thickness and help the platform to get more profits. Let's look at some different approaches of recommendation.

For newly registered users without any user portrait information, the recommendation will face a cold start problem. This is a key question that determines whether the platform can retain new users. There are many ways to solve the problem of cold start. For e-commerce platforms, users can be required to choose their favorite categories of goods during registration which can serve as the basis for coarse-grained recommendation, or just recommend the best selling products of the platform to new users.

If you want to make more accurate recommendations to new users, you can use the method of data exchange to obtain user preferences and behavior information from other platforms that users often use such as social network sites, which is also an indirect user portrait method. The second is recommendation based on the user's historical behavior information, which requires that the user has been using the platform for a period of time and has accumulated a certain number of browsing and purchasing records.

Collaborative filtering algorithm is mainly used here for recommendation. The basic idea of collaborative filtering is to collect users preferences from their historical behaviors and find users with similar preferences or related products for recommendation. The next experiment we will conduct is based on the collaborative filtering algorithm. The third is recommendations based on user portrait. User portrait is a labeled user model abstracted based on the user's basic personal information, browsing records, social activities, and consumption behavior.

Building user portraits is the process of labeling. A label can represent a dimension of the user's characteristics. User portrait usually involves a lot of data processing and feature extraction. Each user is represented as a feature vector which is used to train machine learning models, such as logistic regression to get the use as possible degree of interest in a certain product and then make recommendations. The general process of personalized recommendation is shown in the figure.

The first stage is pre-processing. In this stage, we need to carry out feature extraction and feature construction on the data from various data sources, such as content feature extraction and user behavior portrait construction. The second stage is recall, that is take the features generated by processing as input parameters, train the recommendation model, and use the recommendation model to get the candidate set. The last stage is ranking, adjust the candidate set according to certain rules to determine the final recommendation order and present the final results to the user.