This article mainly introduces the practice and thinking of cloud music in the process of user reach optimization.
I. Preface
At present, the means of reaching users outside the APP station on the market are nothing more than advertising, text messages (phone calls), and notification bar push, and the first two means cost money, and the basic capabilities of notification bar push are basically provided by major service providers for free.
The previous push of cloud music was chaotic, and the business entrances and access platforms were also diverse, and the maintenance was more laborious. All parties feel that their business needs to be pushed and the demand is accessed, but they rarely pay attention to the real effect of the push, and there is a situation where the business is gradually unmaintained but the push is still being sent.
At most, some businesses will count the click conversion of the push as a revenue item, and in fact, most of the people who actively click on the application icon to enter because of Push are counted as organic growth. At the same time, Android manufacturers are paying more and more attention to the management of notification bar push, and the control of push content and allocation limits is becoming more and more strict.
Therefore, we have sorted out and controlled the overall situation, guided by business results, clarified the value brought by push through data, further optimized the current channel capabilities, and achieved significant improvements, providing a lower-cost operation mode for our user recall and frequency improvement.
II. Entrance Alignment
"If you want to do a good job, you must first sharpen your tools", as mentioned earlier, due to the complexity and diversity of business entrances and products, and the two layers of business and channel concepts are not abstracted between products and services, if we do not solve this problem first, we will face many problems in the future and repeat the mistakes of the past, so the technical side first made a more critical optimization here - channel splitting.
Splitting of business and send channels
The business side focuses on the platform capabilities of Push, SMS, and private messages, such as creative (copywriting) management, push plan management, location management, and business-related frequency control, version control, tagging, content classification, and risk control.
Due to the large number of Push products in cloud music at that time, it was time-consuming and laborious to rectify the original products, so we used Link products (cloud music delivery and reach platform) to integrate online and offline, because Push was also aimed at users, and Link's existing capabilities were basically the same, such as creative (copywriting) management, plan management, and at the same time, Push, SMS, Concepts such as private messages are abstracted into the concept of location, so that we have the ability to use a set of delivery products to solve online and offline delivery, and it is based on this that we quickly built Link's offline delivery capabilities in less than a month.
Link is compatible with offline pushes
The above two steps play a crucial role in the governance and optimization of our service platform in the future, and the splitting of channels is equivalent to building infrastructure capabilities, and the offline delivery capability of Link makes product integration possible.
Product integration is to integrate the above-mentioned historical products, unified traffic entrance (here we integrate Push, SMS, private message push on Link products, use Magic Mirror to circle people, use Link to deliver), product integration methods include technology migration, business promotion, communication offline, etc., for example, for some usually less used, usually one or two businesses are in use, try to promote the operation and business migration through communication to go offline, such as personalized Push, There are also some platforms that are no longer used due to business adjustments, such as Ksong's official account push, which can be directly offline after confirmation; There are also some products such as Daystar that are usually not used by many people, and the traffic is not large, but the business coupling is serious, and the business migration is difficult, can be migrated through the technical side, and there is no perception of the business side; Finally, there are some products such as Beijing Operation Push Platform, Nolun, etc., which are still used in large quantities, so it is necessary to leverage the power of the project to come up with a plan to promote the business side to do platform migration。
3. System optimization
As mentioned above, the overall redesign of the Push link is divided into two parts: the upstream is the Push service entrance, including planning/copywriting configuration, policy filtering, document assembly, resource optimization, and circle, and the downstream is the Push channel, which is mainly used to connect with the services of various push channels, including the maintenance of Push device tokens, the distribution of messages to various channels, and the collection of receipt data.
The basic capacity building of the platform has been improved
The basic capacity building of the platform refers to the ability to reuse the online location traffic (splash page, banner, etc.) management with Link, the core delivery and reach platform of cloud music, as the offline traffic entrance, and the ability to build the basic capabilities of Push characteristics in combination with the management related demands of external service providers and internal business parties.
Traffic distribution capacity
At present, Android manufacturers divide notification categories (channels) according to the standard of Google Android O, classify some non-essential push messages into marketing categories, and limit the number of marketing messages per device per day (about 2 messages a day, organized as shown in the figure below), on a first-come, first-served basis.
As a result, the notification bar also needs to have a traffic management allocation mechanism to prevent important operational messages from being preempted by unimportant messages. In this regard, we have made several settings.
1. Messages are managed by setting policies based on the location channel of the link
The main site notification push is divided into two different attributes: notification push and marketing push. Push on the notification bar is applicable to private message notifications, personal high-quality system notifications (such as accounts, funds, etc.), or push reminders related to routine business functions or processes, and the platform does not set frequency control. Marketing Push is applicable to promote applications, content, activities, etc., and frequency control is set up in different channels.
Adaptation of message content classification to vendor classification specifications: Sort out the classification of each vendor, combine the push situation of cloud music, summarize a classification specification that needs to be followed for internal push, and map it with the classification of the vendor. When using it, the business side only needs to select the appropriate vendor classification when configuring the Push plan. At the same time, the operation Push position will only provide the secondary classification of the marketing category to prevent the wrong selection from being punished by the manufacturer.
Increase the ability to control the frequency of the platform operation channel according to the business: The platform operation channel is connected to a large number of business parties, some of which have large demand and some have little demand, so increasing the frequency control capability of sub-service can help control the service usage.
2. Non-immediate notification secondary reach
The click-through rate of general notification messages is relatively high, which is of great help to daily activity, while some non-instant notifications may be regrettable because users miss and do not see them, so add the function of secondary contact to these Push to improve clicks.
3.营销Push分人群频控
In the process of optimization, the data analysis found that some users who like to click on the notification bar message will often click, and the analysis guesses that such users are more concerned about the notification bar and are willing to accept push messages, so the frequency control function of different groups is developed for this part of the user, and more pushes are carried out to improve the activity.
Personalized Push Capability
Personalized Push is a platform operation side proposed to push personalized resources and copywriting according to each person's preferred time, and push all users every day (sharing the frequency control of the platform operation channel) to increase user recall and retention. For the basic capabilities of the platform, it is required to be able to send groups according to the preferred time, configure all the required rules, calculate the data and score the corresponding resources of each rule offline, and prepare the corresponding copywriting online for scoring, and finally the policies will be sorted after frequency control and provided to Link for distribution.
Push open rate increased
The system layer has a switch setting for whether the APP receives the notification bar push, and the user turns off the main switch, which will be divided into several types: some users don't want to receive the notification bar message at all, and just want to use it as a player; Some users just don't want to receive marketing messages, and normal private messages and account-related information still want to receive, but maybe the marketing information is disturbed or they don't know how to close it separately, resulting in all closures; and some users want not to receive push messages during the night period, and turn off the general switch because they are worried about being disturbed during this time。 We did two things to do this:
- The APP side refines the settings of the message receiving switch and provides a Do Not Disturb time period switch
- The system settings separate the operation message, and provide an overlay layer to guide the user to open the system push permission settings when the user opens or has actions such as following the artist
There is a 2-point improvement in the overall open rate.
Optimized push channel capability
Push channel capability mainly refers to the ability to connect with vendors and third-party push services, including reporting, binding, unbinding, and deleting push device tokens, routing and distribution channels for push messages, exposure click jumping, and buried receipts. On this basis, we have made optimization measures to optimize the original message link, so as to improve the overall sending efficiency, reach rate and click-through rate.
Push channel coverage increased
After analyzing the coverage of the original access push channels, we found some places that can significantly help increase the reach.
The first is glory. At first, it was found that some users of Honor's new devices could not receive push messages, and then found that after the Honor manufacturer became independent, it gradually improved its developer ecology, and Honor's system push was also split from Huawei, and the new model began to no longer support Huawei push, and needed to access Honor push. Therefore, we analyzed the push reach rate of Honor and Huawei by manufacturer and brand (as shown in the figure below, data in May). It was found that the reach rate of Huawei's Honor brand was more than 85%, which was in line with the normal manufacturer's channel reach rate, while the Honor brand reach rate of Honor was only 20%+, indicating that only about 20%+ Honor manufacturer's equipment went through Huawei's channels, while others went through non-vendor channels. Moreover, the proportion of Honor devices is increasing, and the official has also begun to promote switching to its own push channel through system updates. Therefore, we believe that the access to the Honor push channel is a significant and controllable improvement in the reach effect, and the priority is increased to complete the access.
Secondly, we found that the reach rate of some devices from non-mainstream manufacturers using Xiaomi's bottom channel push is very high, but Xiaomi push no longer supports non-MIUI devices in the middle of the year, so we pulled this part of the manufacturer's data for analysis (as shown in the figure below). It was found that Samsung, Lenovo, Meizu, Nubia, and BBK had a high reach rate of devices, while Hinova (Huawei Smart Choice) and other devices had a low reach. Because hinova is Huawei's smart choice, it is Huawei's EMUI, which is more in line with the performance of Xiaomi's push on non-MIUI devices. And several other brands with a high reach rate, combined with my previous experience of visiting the flashing forum, guessed that the probability should be caused by the large number of users who brushed MIUI (cloud music does not collect system ROM type data, which cannot be verified).
So here you can see that there are many points that can be improved:
1. Originally, the client initialized the Push channel only according to the manufacturer, not according to the manufacturer's ROM type, all Android other than Huami ov initialized the Xiaomi channel, which just included the device that brushed MIUI, and Xiaomi push was no longer used for non-MIUI devices after 6 months, it was necessary to identify the system ROM type, and more refined MIUI devices were initialized into Xiaomi channels, not just Xiaomi manufacturers initialized Xiaomi channels for devices;
2. Huawei's smart selection of devices also accounts for a large proportion, specifically pulling down the brand distribution of the bottom channel (as shown in the figure below), and found that the proportion of users is not small, if you go to Huawei channels, you can ensure a similar reach rate as Huawei devices;
3. The rest of the devices are connected to the third-party push service provider, using its push keep-alive mechanism, which can be increased to about 10% compared with the original non-MIUI 0.09% of Xiaomi push, which is dozens or hundreds of times higher, and the number of devices from our non-mainstream manufacturers is also millions, and the overall effect is also greatly improved.
Sending failure attribution resolved
There are many links in the entire push link, and there may be failure factors in each link, such as some internal exceptions, some restrictions on vendor interfaces, and device-level frequency control. We have recorded a complete failure cause for each link, and if the reach rate fluctuates, we can analyze the cause and solve it in a targeted manner (as shown in the figure below, we can pull the attribution data of the receipt details).
For example, during our attribution analysis, we found that Xiaomi's manufacturer's interface has a frequency control limit, and the frequency control will be triggered when the Beijing operation is fully sent every day, resulting in about 5 million devices being unable to reach it.
Premium feature support for push service providers
In addition to the basic functions, push service providers generally have some special features to help improve the click effect. If used properly, it can help us improve the reach and retention conversion rate of each push task. For example, the special display will be more attractive to users, and the effect is as shown below. Notification bar space is precious, and the larger it can be, the more effective it is, and of course, to avoid abusive abuse. This type of push style is suitable for some special event scenarios, where the business operator needs to design and customize the event image and use this capability to deliver the large image.
Push device effectiveness maintenance
The push device token is generated by the push service provider, and we associate and save our own user ID and device ID, and when pushing, the user ID can find the corresponding vendor's push device token for pushing. If there is a large amount of invalid data caused by uninstallation and reinstallation, our storage will become larger and larger, and the push efficiency will become lower (a user may find out that only one of the 10 device tokens can be effectively pushed), and the final reach rate effect is not good, so it is necessary to maintain the validity of the device library.
The above figure is the daily user dimension reach rate of each mainstream manufacturer before the complete effectiveness maintenance, which is significantly lower than the actual feeling, such as the OPPO reach rate is much lower than the vivo reach rate, and Apple even cut it in half, and in the test, there is almost no situation where the devices pushed by the manufacturer's channel cannot be received, which is basically the number of invalid devices sent by too many denominators and the overall level is lowered.
At present, there are two main maintenance methods, one is to judge whether the device token is invalid according to the result of the manufacturer's receipt and clean it up, and the manufacturer's receipt is not very comprehensive, and there are omissions, which is caused by this reason. So I made a regular calculation to clean up the equipment that had not been active for a year and had not been exposed recently. The two methods complement each other to ensure the effectiveness of the equipment library. At present, the accuracy of the arrival rate is basically in line with the normal cognition, and the arrival rate of the equipment of the basic mainstream manufacturers is 80~90%. At the same time, it also saves about 40GB of resources for data storage. In addition, the sending efficiency has become faster, and many invalid devices bound to users do not need to continue to push and waste service bandwidth.
Fourth, the strategy is improved
Before elaborating on the specific analysis and optimization ideas, we need to understand the overall logic of cloud music personalized push push.
For a specific user, the decision-making process for the specific copywriting pushed to him on the day is divided into the following three steps:
- Match resources according to the specific rules set by the operation, such as the list of songs that collect the user's hearts in the last 30 days
- Each rule will be configured by the operation of several ideas, using the resources obtained in the first step and the ideas under these rules to combine, can be arranged and combined to obtain a series of final push content
- Optimize the final push content and select the best quality content to push
Among them, there are four strategies that are optimized for the portfolio:
- Random Pick: Randomly select one from all combinations to send
- Resource priority: The algorithm scores all the resources matched by the rules, and selects the resource with the highest score to push (creative random pick)
- Creativity first: There is an algorithm to score all the ideas that can be sent, and select the creative push with the highest score (random selection of resources)
- Resource + Creative Comprehensive Optimization: Combine the scores of 2 and 3 to calculate a certain weighted normalization, and finally return the content with the highest comprehensive score
In the initial stage, since we were uncertain about the effect of the specific strategy, we conducted an AB experiment and evenly divided the buckets for the four strategies
The first stage of optimization: copywriting familiarity optimization
A week after opening the personalized PushH push of all users, we analyzed the return of the first creative data, from the overall market data, the average click-through rate of all the creatives configured is not high, this value is lower than expected before the start of the project, so we ranked all the creatives according to the click-through rate, and conducted an in-depth analysis, and soon had a new idea, dozens of rules listed in the operation have a clear recommendation logic, so you can improve the user's familiarity with the copywriting by adjusting the copywriting in the creative and highlighting these elements。 According to this idea, after the optimized copy, after a week of distribution, most of the optimized copywriting click-through rates are higher than the original copy, and some specific copywriting has more than doubled compared to the original copy!
The second stage of optimization: creative distribution strategy optimization
After optimizing the ideas of the first stage, we have obtained some high-quality ideas with high click-through rates, but from the overall market, the total daily click-through rate and the number of click-through users of Personalized Push have not increased much, so we started the second round of data analysis and optimization
In the previous stage, our focus was mainly on the click-through rate of the creative, and we ignored the impressions of the creative, and by listing the impressions, we quickly found the problem: the poor click-through rate of the poor accounted for the majority of the overall exposure, while the high-quality creative, although the click-through rate was higher, only a small part of the exposure, so the overall market click-through rate and total click volume were very low.
As mentioned above, at the start of the project, a total of four different combination optimization strategies were set, namely random (for control), creative priority, resource priority and comprehensive priority of creative resources.
For more than a week, the effect of using algorithm-optimized bucketing (whether it is creativity or resources) is only the same as that of the random distribution strategy, which is obviously not in line with expectations, so we immediately conducted a link investigation and optimization discussion with the relevant algorithm students, and soon we found a series of problems:
In the implementation of the resource algorithm, due to the direct reuse of the algorithm model of private FM, the corresponding optimization direction actually pays more attention to the songs that "users may like", and this model will give priority to recommending more "fresh" song resources to users, but this direction is contrary to the previous analysis, users need more familiar resources, so the algorithm model should be targeted for the Push scene, and the resources that users are more familiar with are optimized.
In terms of the implementation of the creative algorithm, it was found that the model did not take the click-exposure return data of the creative as the input of optimization, but only scored each copy according to the existing original data set, so it did not play a real role in the Push scene.
After the adjustment of the algorithm model, the click data of the creatively preferred bucket began to climb immediately, and continued to be significantly better than that of the random control bucket. After a period of observation and analysis, we judged that creativity has a greater impact on users in the entire push link, so we expanded the bucket using the creative priority combination optimization strategy, from about 20% to 65%, and after the expansion, the click-through rate and click-through data of the dashboard quickly changed, and the click-through rate increased by nearly 80%.
Looking at the distribution of creative exposure/click ratio at different click-through rates, it can be seen that high-quality creatives with higher click-through rates have significantly received more exposure and brought more clicks:
At this point, the analysis and optimization of the second stage was basically completed, and in this stage, we optimized the distribution strategy of the idea with the help of algorithm capabilities, and finally achieved the simultaneous growth of the click-through rate and total clicks of the market
Phase 3 optimization: Increasing the supply of high-quality ideas
After completing the second phase of optimization, we basically realized the discovery of high-quality ideas and traffic skew along the entire chain, but with the increase of delivery time, we found that the overall click volume of Personalized Push fluctuated significantly, and there were often some peaks and troughs.
In the creative optimization mode, the clicks are mainly driven by high-quality creatives, so we focused on drilling down to analyze the exposure and click behavior of the top high-quality creatives, and found that there is a relatively obvious head effect of high-quality creatives, among which the three most effective creatives have contributed to an astonishing 30% of the total number of clicks , and because the idea itself has frequency control (in order to prevent user fatigue, it is set to once every 3 days), so whenever these head ideas are frequency controlled, the number of clicks on the overall market will drop significantly, and when the frequency control is over, the number of clicks on the market will rise rapidly, which will cause "peaks and troughs" on the data.
The reason for this phenomenon is that the supply of high-quality ideas is insufficient, and only single-digit high-quality ideas can bring good clicks in the overall more than 500 ideas, and the system cannot supplement the same high-quality ideas after these head ideas are frequency controlled, and the traffic can only flow to relatively mediocre ordinary ideas, resulting in a decline in click-through rates; 。
In the first stage, we optimized the copywriting in the direction of "familiarity", and although the click-through rate of the optimized copywriting in this stage has increased to a certain extent, most of them are still good and medium-level creatives, and do not produce enough high-quality head ideas. To this end, we have conducted a comprehensive analysis of the top 20 ideas with the highest recent click contribution, trying to find the creative ideas behind the high-quality ideas, and then expand more high-quality ideas.
Through the analysis of the head ideas, we quickly found the commonalities of some high-quality copywriting, in order to assist the operation students to quickly diverge and produce more copywriting and improve the overall supply, we also combined the hot AIGC technology, through the way of small sample prompts, quickly produced hundreds of candidate new ideas from three different directions, and then through the operation of secondary processing to produce the final ideas that can be used for delivery:
After adding this part of the new creative supply, the daily exposure trend has been significantly improved, and the original head idea can still have enough high-quality creativity to be exposed after the trigger frequency, and from the perspective of continuous delivery, after the exposure of the high-quality creativity on the previous day, there can be other high-quality creativity to fill in the position on the next day, and the number of clicks on the market no longer only depends on a certain head creative.
In addition, among the newly added ideas, "explosive models" were also born, for example, the following creative push received a very high number of user clicks, and the total number of clicks on the market reached the peak since the project was launched:
At this point, the third stage of optimization is basically completed, in this stage, we analyze the composition of high-quality ideas, refine the selling points, and use AI to assist in the production of new ideas with similar ideas, so as to stimulate the supply of high-quality ideas, while driving the number of clicks on the market, it can also effectively avoid the fatigue of users to Push copywriting.
Optimize your analysis
After nearly two months of creative analysis and optimization of Personalized Push, we doubled the overall click-through rate of Personalized Push from an average at the beginning of the project. In this process, data-centric, continuous observation and drill-down analysis of the effect of creative delivery, as well as further decision-making and optimization actions based on the analysis results basically constitute a closed-loop model:
- Perception stage: Rich and comprehensive data reports are required, as well as drill-down analysis based on various dimensions, such as the overall click and exposure distribution of the creative, the influence of different dynamic variable factors (song title, artist name, etc.) in the creative, and the effect of a single creative on different groups of people (age, region, activity, etc.)
- Decision-making stage: According to the data analysis of the previous stage, find out the problems or potential optimization points on the current delivery, and decide which direction to optimize and adjust. The decision-making stage is usually the precipitation of past optimization experience and methodology, for example, if you find that the overall click of the idea has a large head effect, you should adjust the supply, if you find that the exposure of high-quality ideas is insufficient, you should adjust the distribution strategy, and of course, you can also carry out some innovative explorations, such as optimizing the expression form of the idea (adding pictures, etc.)
- Action stage: According to the decision, the actual implementation of specific actions, such as adding ideas, adjusting algorithm models, operating intervention and distribution rules, etc., often needs to carry out AB experiments and other operations in the action stage, and in this stage, it is often necessary to reduce the operation cost of operation through excellent product mechanisms.
- Feedback stage: The delivery data generated by specific actions is fully recovered, and it is returned to the data report and data analysis in a timely manner, so as to form a closed loop with the perception stage.
5. Summary and outlook
After a series of optimizations, the overall conversion funnel increased significantly (the number of users decreased due to the removal of invalid devices by device effectiveness maintenance), and the number of users who clicked on the final account almost doubled. In addition, the analyst attributed the number of users who saw the push directly click on the desktop application icon to initiate, further affirming the business value brought by the push.
The integration of the link entry into this optimization to system optimization and then to policy improvement is very clear, and each stage provides a solid foundation for the optimization of the next stage, which is also the main factor for the project to achieve great business results. Looking forward to the future, we can precipitate and polish a set of systematic, data-driven operation mechanism based on the system foundation and experience model, and continue to drive the effect of Push.
Author: Zhu Mingming, Jiao Guangcai
Source-WeChat public account: NetEase Cloud Music Technical Team
Source: https://mp.weixin.qq.com/s/6P4dmPFe_HW1PfOaqcufeQ