This article starts from the H5 business of the marketing channel, describes the general testing skills and implementation solutions of H5 testing, and summarizes and refines the testing work and experience. Through this article, you can understand the test methods and practical effects of the commonly used R&D architecture in JD.
【Table of Contents】
- Learn about H5 business
- Commonly used testing methods
- Tests on JD's existing H5 common architecture and implementation schemes
- Pain points and deficiencies
background
H5 page is the most common form of operation in the marketing domain, and the business provides services through H5, which can meet the needs of users for convenience, efficiency and low cost. The H5 page is the endpoint of the business to the user, and its quality assurance work is particularly important. The functional implementation of each business is universal, and there are corresponding common test methods, which are summarized and shared in this article.
1 Understand H5 business
Understanding H5 services from multiple perspectives, understanding the implementation link of functions, and clarifying which party implements each node can open up the idea of designing use cases. On the other hand, when encountering problems, it can be quickly located and accurately feedback.
[1.1 Front-end display]
1.1.1 Two development technologies
When it comes to front-ends, we need to first introduce two development technologies: "native development" and "H5 development":
Native app development: It is to use the official development language, development library, and development tools on mobile platforms such as Android and iOS for application development. Therefore, the native architecture of the app should be the best in terms of application performance and interactive experience, such as the "live stream", "login" and reminder components in the APP, which are purely natively developed modules.
H5 development: refers to application development using web technologies (HTML5, JavaScript, CSS). The benefit of H5 development is that it can be cross-platform, and the code written can run on Android, iOS, and Windows at the same time. At present, the main activities in the APP, such as "10 billion subsidies", "cheap free shipping" and "seckill", are all developed and realized by H5.
The comparison of the characteristics of the two development implementations is as follows:
| H5 | Native | |
| Development costs | Low: One set of code, cross-platform use | High: You need to write two sets of the same logic and interface |
| Development cycle | Short: The magnitude is low, and the function is directly added and released | Long: The update iteration is slow, and you need to wait for the official review when it is released |
| Invoke the underlying function | Complex: It cannot be called directly, and other operations such as bridging are required | Simple: It is closer to the bottom layer, and it is also easy to retrieve the underlying functions |
| Performance experience | There are limitations: The H5 mobile app doesn't have direct access to device hardware and offline storage | Better: Runs directly on the device's operating system and typically performs better and responds faster |
| Deploy updates | Fast: Just update the code on the server, and users can enjoy the latest features without downloading | Slow: Releases and updates need to be made through the app store, and updates may require users to download a new version |
| marketing | Flexible: It can be promoted more easily through channels such as websites and social media | More restrictive: Promote and distribute through the app store, but you need to comply with the store's policies and guidelines |
1.1.2 Container angle layering
Taking the Spring Festival Gala page as an example, from the perspective of containers, an H5 page is displayed from the top to the bottom level as shown in the following figure:
Among them, the Babel H5 fragment container rewrites the CommonMFragment of JDHybrid, and the X5WebView container rewrites the x5webview, allowing you to decide whether to use the system or x5 at your own discretion. There are four parties involved in the figure, namely the Babel team, the JDHybrid team, JSSDK, and the H5 specific business parties. Among them, the Babel team and the JDHybrid team are native development architectures and belong to the container side, while JSSDK and H5 specific business parties belong to H5 development, and their respective roles can be summarized as follows:
•JDHybrid: provides environment device information, navigation bar, page routing, page events, general JS functions, and performance optimization
• Tower of Babel: Provides custom navigation bar logic, including UI and JS bridge; Other capabilities to reuse webview containers
•JSSDK: unified API, call client protocol; At the same time, it provides performance exception reporting and common functions
•H5 front-end: access JSSDK, display page content, implement front-end interaction and other business logic
1.1.3 Implementation of specific functions
The implementation of specific functions often involves multiple function providers, which can be roughly divided into two categories:
• Competencies are provided by JDHybrid
H5 adjusts the JDHybrid encapsulation method through JSSDK, and JDHybrid adjusts its own logic
For example, you can obtain the UUID in the device information
JDHybrid provides a JS bridge to obtain basic device information, and you can enter parameters according to the agreed rules to obtain information such as uuid. However, the native underlying API, but no longer exposed to the outside world, but maintained by JSSDK in a unified manner, JD Electric's H5 code only needs to call JSSDK
• Abilities are provided by other teams (Babel or other components).
H5 tuning JS code, through the webview kernel, the kernel calls the unified method of JDHybrid encapsulation, JDHybrid tuning Barota (or other plugins)
For example, if you open an address list, the address list is a capability provided by the address component, and JDHybrid provides a routing method that can be used to test the demo to determine whether the specific business problem is a specific problem.
[1.2 Content data]
Applications that can provide the same type of functionality for multiple businesses are abstracted into "upstreams". From configuration to presentation to users, marketing content needs to be processed by multiple business logics, and in addition to refined business processing on the business server, it also needs to interact with various upstreams. The following figure shows the links between the function of a service as a whole and the common upstreams:
1.2.1 Data Sources
Product information, coupons, red envelopes, and benefit points are common elements of an H5 page, and their underlying sources vary:
| Page elements | Underlying sources | Example |
| Product information | Target product groups | |
| Coupon information | Coupon in the middle of the platform | |
| Red envelope information | Red envelope in the middle of the platform | |
| Profit points | Business CMS | |
1.2.2 Data Strategy
In general, the business side does not directly interact with the underlying data, but uses multiple upstream to achieve the effect of the data in thousands of ways, such as:
•Algorithm: According to the business configuration strategy, the product group information is integrated and provided to specific services
•UMC: Based on user data, formulate rules for different types of benefits for different groups of people
•Interactive Workshop: Set the rules for the combination of tasks and prizes according to the activity dimension
Regarding the verification of content data, the key to testing is what you match is what you get, and the data obtained by different user portraits must meet business expectations.
2 Common test methods
[2.1 Test server]
2.1.1 View logs
• Platform: Tarzan - Log Management
•Applicable scenario: upstream and downstream logic is involved, and cannot be directly observed on the front-end
• Concerns:
You can filter the information of each application by keywords to verify whether the input parameters from the server to the upstream and the return from the upstream to the server meet expectations
2.1.2 Special Scenarios
•平台：deeptest-mock管理
•Applicable scenarios: For some abnormal scenes or boundary values, marketing activities or materials cannot accurately meet the requirements of the scene.
• Concerns:
You can enter upstream interface information on the platform and return it through mock upstream to verify the processing logic of the business server
2.1.3 JMQ Validation
• Platform: Tarzan-JMQ
•Applicable scenario: the scenario in which MQ is used to communicate between application services
•Focus: Enable consumption tracking to verify whether the MQ messages sent to other application services are accurate and the content is correct
2.1.4 Caching Queries
• Platform: Tarzan-JIMDB
•Applicable scenarios: The requirements are changed to caching logic, especially for long-term interactions
•Concerns: whether the cache is written in a timely manner, whether the validity period is reasonable, and whether the cached content is correct
2.1.5 Direct calls
•平台：deeptest-用例管理
•Applicable Scenarios:
The pre-operation is long (such as need to display before collection), the conditions are harsh (such as the need for multi-identity marking), and the threshold is high and batch operation is required
•Concern: The interface returns the same input parameters as expected, and the boundary logic is handled correctly
[2.2 Test front-end]
2.2.1 Functional Testing
| function | Verification points |
| User Conduct | click Single click: whether the click event is responded to, whether the multi-layer page will be clicked through multiple clicks: whether the subsequent click behavior will cause business logic errors while waiting for the data to be returned on the page |
| slide Sliding speed: different speed sliding, business functions need to be consistent, and fast sliding data loading should not be too slow Sliding interaction: whether it supports left and right horizontal swiping and whether it responds to click operations when swiping | |
| flushed Active refresh: Click the button to refresh by dropping down the button, pay attention to page loading behavior and API request passive refresh: business-specific logic, and pay attention to triggering refresh timing and interaction | |
| System interaction Return: return to the first-level page and return to the second-level page, pay attention to the return level and historical browsing history Input: special content, formatting, input panel evoking and hiding Retreat before and after the backend: whether page behaviors such as countdowns, animation effects, interface requests, etc. are interrupted | |
| Multimedia-related | Image Image display enlargement, restoration, switching and other operations are supported |
| Audio and video Resource loading under different domain names Interactive experience: play, stop, exit | |
| Page requests | Check and modify the requests and responses of HTTP, HTTPS, and Websocket to verify whether the front-end input parameters and various data display logic meet the expectations Interface Request Animation effect in the process of API return Request timing, interface degradation, and interface abnormality front-end support |
| Resource requests Whether the request is repeated The correctness of the requested data in the scenario of page turning and pagination is made | |
| login | The path of the user who is not logged in Login status is opened up Identification of different users Nickname and avatar display of special accounts Use WeChat or other off-site information to log in |
| Pop-up box | When the pop-up box is triggered Correctness of the content of the pop-up box The material and animation of the pop-up box The trigger conditions for closing the pop-up box |
| Network environment | Different network environments: WiFi, 3G/4G/5G network environment switching, whether users have perception User experience under weak network conditions No network bottom line logic |
| Thorough testing | For key fields, verification such as "not delivered", "empty", and outliers is used to avoid user-perceivable problems such as "opening the skylight" and falling off the building caused by abnormal delivery |
2.2.2 Compatibility Testing
Coverage Principle:
•Android, iOS different systems
• Take into account different screen resolutions
•If off-site delivery is involved, it is necessary to consider that the container version is compatible with the WeChat version and different native browsers
•System kernel, X5 kernel
Platform:
At present, some automation methods, such as Airtest and activity automation testing, are integrated into the Cybercloud testing platform in the form of plug-ins
2.2.3 Buried point test
Concern: whether the name of the buried event, the timing of the report, and the key fields are consistent with the tracking scheme
Platform: track
2.2.4 Integration with native architecture
| function | Verification points |
| APP VERSION | It is necessary to pay attention to the minimum version supported by the component/framework, carry out version control, and boundary testing • It is necessary to distinguish between native clients, iOS, Android, and HarmonyOS for functional verification |
| Fault-tolerant means | APP changes need to be re-released, and those that have already been released cannot be used, so pay attention to verify the logic of the function switch, configure and improve the downgrade method, such as downgrading some functions according to URL parameters |
| Kernel compatible | System kernel: This can be simply understood as the browser kernel, also known as the rendering engine or typesetting engine, which mainly interprets the syntax of the web page, renders the web page, and converts the code of the web page into a visible page. X5 Kernel: Originally optimized by Tencent based on the open source Webkit. Based on the X5 kernel, the TBS service provided by Tencent integrates Tencent's underlying browsing technology and Tencent platform resources and capabilities to provide an overall browsing service solution. Therefore, in the Jingdong APP, after the APP is installed, the X5 kernel will be downloaded for H5 use |
Caution:
1.X5 kernel needs to open the storage permission for JD APP before it can be downloaded
2. After the X5 kernel is downloaded, you need to restart the APP before you can use it
3. Quickly locate the problem: Use the browser that comes with the mobile phone to visit the H5 page, if it is inconsistent with the performance in the APP, you can narrow the scope of the problem
[2.3 Online Tracking]
After the requirements are launched, they need to monitor, analyze, and verify the functions, performance, and experience of the requirements in real user scenarios. At present, the company's existing tracking platforms and methods are displayed as follows:
| platform | Concerns |
| Voice of the user | Real user feedback, with a focus on user experience |
| Xingyun-interface monitoring | The business logic processing of the monitoring interface focuses on the connectivity and availability of business services |
| Tarzan - radar screen | You can observe the health status of system services from a global perspective, focusing on the performance of full-link services |
| UIπ - Woodpecker | Detect all kinds of problems on the H5 activity page, focusing on front-end display |
| Candle Dragon | It can provide multi-dimensional user behavior information, which is helpful for troubleshooting user problems |
3 Tests on JD's existing H5 common architecture and implementation schemes
[3.1 Release]
3.1.1 iHub
The large front-end co-construction platform is built based on iPaaS standards, and provides developers with cross-terminal floor development and management capabilities including h5, iOS, Android, etc. Empower developers to reuse, retrieve, and redevelop development content across business lines and systems (in line with iPaaS standards).
Verification Points:
•Location: The floor is above the fold, not above the fold, etc., verify whether there are any abnormalities, such as data loading, floor rendering, etc
•Quantity: Whether multiple co-construction templates are used on a page and whether there are conflicts
•Coexistence: Whether there is any abnormality when the co-construction template coexists with the Babel's own template
•Linkage: Anchor navigation for co-building template associations
At present, an automated testing method for the promotion venue has been precipitated
3.1.2 Tower of Babel visualization platform
It is a visual platform for building activities/channel pages, which supports the construction and output of H5, native, PC and other multi-terminal pages at one time, which is free for users such as production and research, procurement and sales operations, and merchants
Verification points: Visual configuration, server-side saving and delivery, front-end display are normal, and pay attention to the compatibility of new function points with old functions
[3.2 Performance Optimization]
It is particularly important for users to be able to access the page normally for the content of the page to generate value, minimize the loading time of the page, and then reduce the bounce rate. At present, there are some relatively mature performance optimization tools in the company, which will involve the testing and verification of tool access and effect.
3.2.1 Page Loading Process
The loading process of an H5 page can be simply summarized into the above steps, and the performance optimization methods are mainly from the aspects of early request timing and reducing resource requests.
3.2.2 Existing optimization methods
Verification principle: The access takes effect and has no impact on the service logic after the access.
3.2.2.1 JDHybrid离线包
Principle:
Package some static resources (such as img, js, css, html, etc.) above the fold to the local disk in advance, and get the resource load directly from the local disk (or memory) when the page is loaded
Verification Method:
1) Logs: With the help of the testing tools provided by the JDHybrid team (Xconsole, xdog, etc.), confirm that the corresponding resources use offline resources
2) Packet capture: H5 does not initiate network requests when using the resource
3) Hybrid Rapid Verification Tool:
Usage:
Babel Tower venue, cross-night evening, Spring Festival Gala, etc
3.2.2.2 Tower of Babel - Direct data out
Principle: The front-end obtains the display data directly from the HTML without initiating a request for the above-the-fold interface.
Verification method: Observe packet capture and do not initiate network requests on the connected floor
Services: Some Babel venues, coupon collection centers, etc
3.2.2.3 通天塔-SSR
How it works: The server renders the content of the web page and sends the rendered HTML to the browser, which displays it directly. The difference between data ejection and SSR is whether it loads an entire html directly, or loads a slice of pages in the order of pages first and then floors.
Verification method: Disable JS, the page can still be loaded
Use business: tens of billions of subsidies, cheap free shipping, etc
3.2.3 Verification of optimization effect
3.2.3.1 Peer comparison performance test tool
Record the user's operation process, and collect and analyze the time-consuming scene from the user's perspective by automatically splitting frames. In the case of controlling variables, the performance can be compared and analyzed with competing products
3.2.3.2 Candle Dragon
Through the intrusive burying method, the all-round monitoring of APP applications is realized, real-time collection of users' performance anomaly data, rapid and accurate positioning of problems, discovery of performance bottlenecks, reduction of user churn, and improvement of user experience
[3.3 Risk Control]
The commonly used risk control methods in H5 focus on the two directions of anti-crawling and user identity, and the focus of verification is on "the correctness of access" and "the effectiveness of the policy".
3.3.1 Three-piece set of price anti-climb interfaces
• Aegis disposal
Verification Points:
Login to the blacklist and whitelist, request the interface, trigger disposal, and the gateway returns 605
If you abandon the verification on the disposition page, you can return to the previous page and cannot recycle to the disposition page
The disposition page is successfully verified, the disposition page disappears, and the H5 page is reloaded
• Aegis interface reinforcement
Verification Points:
The H5ST parameter is normal, the signature verification panel returns 200, the SOA interface delivers data normally, and the frontend page is displayed normally
If the mock input parameter is abnormal, the verification panel returns a non-200, the SOA interface is intercepted on the gateway side (403 or mock data is delivered), and the frontend page follows the business bottom logic
• Device fingerprint
Verify that the parameters in the body are correct
3.3.2 RCS Risk Control
Verification point: Verify whether the service processing logic triggered by the corresponding PIN meets the expectations based on the configuration policies of different portrait groups
4 Pain points and shortcomings
1. Component testing
The code changes of the component are low-level, and the testing process is relatively black-box, and when the test scope is delineated, it can often only be repeated regression, because the more low-level logic cannot be measured, such as the scene cannot be created. How to improve testability and increase test accuracy is a problem that needs to be further solved.
2. Compatibility Testing
At present, there are many device models, and when it comes to compatibility testing, it is actually a repetition of a single behavior, which takes a long time to perform manually and has limited coverage. However, there is currently a lack of reliable automation tools that can replace compatibility verification while reducing the maintenance cost of scripts.
3. Test the footage
When it comes to equity-related requirements, relying on real material can hinder the progress of testing. The premise of using mock is that one party guarantees it, or the content has been verified, which is risky.
4. Thorough testing
In large-scale interactions, there are many calling interfaces and complex interactions, but the requirements for robustness are high, and the workload is large. The current automation tools also need to be supported in a variety of scenarios.
Author: Technical Quality Ding Jiaying
Source-WeChat public account: Jingdong Retail Technology
Source: https://mp.weixin.qq.com/s/AlwRQriRqVfGmWC8sgRq7w