Editor's Note: The Internet has made the dissemination of information from the traditional print media to today's decentralized UGC method. When massive amounts of information are distributed, as product designers, the problem we need to consider is how to do a good content distribution system. Today's headlines provide us with a direction - algorithm recommendations. So, as a new product, how to complete a recommendation system from 0 to 1, the author summarizes four steps to share with you.
The Internet has gradually transformed information dissemination from the traditional centralized paper media into a decentralized UGC method. In this era, everyone can be an information producer, an information disseminator, and even more an information consumer.
And when massive quantities are produced and information is overloaded, how should we distribute and consume content? Zhang Yiming and his today's headlines give us a scheme - algorithm recommendation. How to do a good job of algorithm recommendation is also regarded as the most challenging thing in the product.
So as a new product, how should you complete a recommendation system from 0 to 1? I will explain it to you in the following four steps:
<h1 toutiao-origin="h2" > first, product attribute analysis</h1>
The first thing you have to understand is that not all products need to do recommendation systems, and the recommendation strategies of different products are not consistent, after all, every algorithm engineer is a mobile vault (==), and the cost of a good recommendation system is quite high.
Understanding your product attributes and user needs is the most important step. It is generally believed that products with large resources need personalized recommendations, such as Taobao, Douyin, Sina News and other information distribution products, while tool-based products such as WeChat and WPS rarely need to be recommended. We divide product attributes into user attributes and resource attributes:
<h2 toutiao-origin="h3" >1</h2>
We need to understand the user composition of the platform and what they need, for example, if the user only wants to edit the document with your software, then it is useless for you to recommend more videos for him.
<h2 toutiao-origin="h3" >2. Resource attributes</h2>
Resource attributes refer to the resource composition of the platform, that is, what your products have, which can be virtual products (high-quality articles, Q&A, etc.) or physical products (mobile phones, audios, etc.).
And when these two are combined, different users need different resources, then we need to recommend strategies.
If you can understand this part, you can also roughly understand why most tool-based products don't need to be recommended. Tool-based products in order to ensure product depth, most of the functional requirements provided are convergent and single, most users use the same service, so there is no personalized recommendation.
Do you think product attribute analysis just lets you know about recommendations? No, in fact, product attribute analysis has greater value, because it determines the specific goals of the recommendation strategy. For example, if you are a video site, the goal may be to increase the user's viewing time. In the entire algorithm strategy, it is necessary to split and optimize around this viewing time. If it is an e-commerce platform, it is necessary to optimize the recommendation strategy around the amount of orders placed.
<h1 toutiao-origin="h2" > second, feature engineering (labeling system).</h1>
Let's go a step further, we must lay the groundwork before we can develop a recommendation strategy. We need to understand the more specific needs of users, and we also need to understand what resources the platform has. In this way, it is possible to achieve the matching of users and resources, a process we call labeling, and in practice also known as "feature engineering".
There are many types of tags, and conceptually we are mainly divided into two types: "user tags" and "resource tags".
<h2 toutiao-origin="h3" >1. User tag</h2>
The general user tag includes three types: basic attributes, active attributes, and interest tags:
Basic attributes often refer to the user's natural attributes such as "gender", "age", "permanent residence", "mobile phone device model", "professional level", etc., which are the basic attributes that users objectively possess when they do not use the product.
The active attribute refers to the behavioral data left by the user when using the product, and is divided into two types: statistics and rules according to the calculation method. The statistical category refers to the data that can be directly calculated statistically, such as the number of active days of users, the cumulative amount of payment, the number of activity participation, etc.; while the rule label refers to some relatively complex labels, which need to be calculated for the first rule model. For example, user activity levels (high, medium, low), user participation intentions, etc. need to be clearly calculated in advance. User activity data is also often used to assess user stickiness and cycle value.
The interest tag belongs to the mining class tag and generally relies on the resource tag. It means that when the user browses the specific content resources, the tags of the resources themselves are attached to the user, and the more the user uses the behavior, the more and more accurate the interest tags. And according to different behaviors, the score of each label is also different (for example, the search as a user takes the initiative to put forward the demand, its score will be higher), the more you look at the vibrato, the more you want to see it, the same is the same. According to the application scenario, it is divided into short-term labels (online calculation, generally within 2 days of labels) and long-term labels (offline calculation, referring to historical cumulative labels).
<h2 toutiao-origin="h3" >2</h2>
Refers to the tags of various resources in the product, divided into two dimensions: "category" and "keyword".
Category tags are classified by some specified rules, generally according to the complexity of the resources divided into 2 to 5 levels, there are also platforms with more levels. The following illustration shows the category label for a lobe.
Keyword tags are a thinner layer on top of categories, referring to specific tag words. For example, when users are interested in politicians, we find that it is mainly reflected in the name of "Trump", and some business information about Trump can also be appropriately recommended.
Through feature engineering, we tag each user and resource with a large number of labels before introducing a recommendation strategy. These two types of resource tags are generally added through manual labeling and machine learning. However, machine learning requires a large amount of labeling to achieve a certain degree of accuracy, so it will rely more on manual labeling and thesaurus expansion in the early stage of the product. After reaching a certain data scale, train machine learning. Finally, through continuous machine learning + manual correction, the entire feature engineering can achieve a certain recognition accuracy.
< h1 toutiao-origin="h2" > third, recommended strategy</h1>
When we collect the user tags and resource tags, the next step is the recommendation strategy, which is generally divided into two modules: recall and sorting.
<h2 toutiao-origin="h3" >1</h2>
When users access the product, we give priority to recalling resources that meet the user tag from the resource library, which is usually the order of thousands / millions of data, and then sort the display according to the tag matching degree of these resources, time, etc. Mature products will also involve fine and rearrangement, according to the user's use of each resource, real-time change the ranking of subsequent resources. In practice, engineers will encapsulate the recall and sorting into a recommendation engine, and then each internal link will have the corresponding algorithm personnel to follow up the optimization, which is the so-called tuning parameter.
<h2 toutiao-origin="h3" >2. Recall/sort specific strategies</h2>
Recall and sorting are two rather large projects in the recommendation algorithm, involving many methods, and here is only a brief sharing of the main strategic components.
1) Recall
There are thousands of resources in the resource library, but only a few dozen or even a dozen are displayed to the user in the end, and if you directly calculate and sort all the materials, it is not only extremely costly and poorly responsive. Therefore, we need to conduct a preliminary screening of the material and recall a batch of candidate sets that may be of interest to the user. The traditional standard recall structure is generally a multi-channel recall, mainly divided into "personalized recall" and "non-personalized recall" two categories, personalized recall pointer to the user feature recall, mainly "interest tag recall", "collaborative filter recall", etc.; non-personalized mainly refers to "hot recall", "cold start recall" such as unified features of the recall.
In practice, one or more of the above recall strategies will be selected according to different scenarios. For example, in the search scenario of recall sorting, after you search for a certain product on Taobao, you will find that the product ranks first when you visit the list again. The diversity of recalls is important, and sometimes the effect of more recall strategies may be amazing, and the quality of recalls also largely determines the upper and lower limits of the recommendation system.
The following figure shows a schematic of the "Collaborative Filter Recall":
2) Sort
Generally when the candidate set reaches the order of magnitude of "thousand", we begin to need the sorting strategy, generally through rough and fine row to reduce the candidate set to the "hundred" level and score, sort by the top score, and then reorder according to the user's real-time feedback, the amount of data is reduced to the "ten" level for sorting display.
The goal of sorting is to change according to the business goals, and in the early days, due to the simplicity of the business goals, when it is necessary to focus, it is often necessary to choose an indicator to focus on optimizing the sorting. However, with the increase of multiple recall strategies, it will be found that the overall improvement of a single indicator is very limited in the medium term. At this time, we need to introduce multi-objective sorting to solve these problems, such as combining time, interest, hotspot, location and many other dimensions of data for comprehensive sorting, here because we need to pay attention to different user scenarios and their sorting focus is different, so different sorting strategies are needed to improve accuracy. For example, more attention is paid to interest tags in the interest stream, and more attention is paid to interactive data in the hot information stream.
The commonly used sorting algorithm framework has three categories: pointwise, pairwise, and listwise, x1, x2,... Represents training samples 1,2,... Features, y1, y2, s1,... etc. is the label (objective function value) of the training set. Interested students can delve into the following on their own, which will not be repeated here.
<h1 toutiao-origin="h2" > fourth, continuous optimization of the model</h1>
When a recommended model is built around people and things, the work is not over. Instead, it's just getting started.
Most products need to face more complex policy rules when they are first recommended, but due to the lack of actual data support, it is often the algorithmians who give the initial model based on personal experience and competitors as a reference (such as the initial policy setting hotspot weight 4, time weight 2, interest weight 2, etc.) in a feed), so the effect is also uneven. Generally, after the grayscale is online, the actual user feedback data (such as ctr, completion rate, order rate, etc.) is required to continuously tune and improve the model.
We mainly divide the data validation for a model in two main steps:
Offline evaluation: In the offline prepared new data set and the previous model data to compare, such as accuracy, coverage, diversity and other aspects, such as the model's comprehensive indicators are better than the online model, you can enter the online experiment, here is generally checked by the product manager.
Online experiments: When the evaluation model is better, we need to conduct A/B bucket experiments online. The general experimental cycle is about 2 weeks, comparing the experimental group and the control group, if the experimental data is positive, it is pushed to more users to continue monitoring. If it is reversed at a certain stage, the optimization model strategy is returned, and the experiment is repeated until it is gradually pushed to the full number of users.
The phrase "there is no best, only better" is also a true portrayal of the recommendation system. Algorithmic recommendations are not like other functional requirements, and it doesn't have to be absolutely completed that day. The algorithm system as strong as Toutiao and Douyin has been so sound that it still spends a lot of money every year to recruit algorithm posts. Because with the development of society, user habits and hobbies always have different degrees of change. Therefore, keeping the recommendation system agile, long-term continuous monitoring and strategy optimization is the norm in the entire recommendation system.
Today's end here, divided into "product attribute analysis", "feature engineering", "recommended strategy", "model continuous optimization" four parts, limited space details part is ignored, I hope you include more. I believe that with your efforts, more complete and imaginative recommendation algorithms will continue to emerge and prosper!
This article was originally published by @Xu Mu and is not reproduced without the permission of the author.
The title image is from Unsplash, based on the CC0 protocol.