Conception

As an interface automation test engineer, the most common task you face on a daily basis is to write interface automation test scripts, so what are the most boring and boring things in the process of coding?

Pain points

Every time we get a new interface, we have to manually refer to the documentation to generate an interface class in the script, and the more parameters it takes, the more time it takes
The requirements are different, but robustness and repetitiveness for some business use cases are high
When you want to refactor scripts, the sheer amount of writing of interface data and use cases can be daunting

Spending 30% of your day writing scripts that don't require much thought is not automated!

solution

Parsing the document
Sort out scripts that are suitable for auto-generation
Use the tool to generate this part of the script

Expected goals

Free your hands and reduce the proportion of pure manual labor, which in turn gives yourself more time to think, understand the product, and design more "smart" use cases

Practice

Automatically obtain API information

Analyze the structure and content of interface automation scripts

Analyze the scheme of automatically generating test scripts

Diagram of the automated test script

Filter scripts that are heavy and have a pattern to follow

The rules here should not be too complicated, you can choose the part with simple logic first, we mainly choose the following two parts

Interface, working time accounts for 30%~50%, characteristics: specific structure, data from other platforms

Diagram of the interface class

In the use case part, the working time accounts for 30%~50%, and the characteristics are: the repetition is higher than about 80%, and the generation logic can be described

Use case structure diagram

Parse the interface documentation

Interface information comes from interface documents, and the more mainstream interface document management tools on the market are Swagger, RAP, WIKI or other common document tools.

The following analyzes and compares the differences between the following tools for the purpose of parsing interface files:

classify	Swagger	RAP	WIKI
description	A framework for generating, describing, invoking, and visualizing RESTful web services	Visual interface management tools	A hypertext system that can be used by multiple people to collaborate on the author
format	json	json	html
specification	There are unified specifications for the specific structure and type of each parameter and return value	同swagger	You need to agree on your own specifications
cost	Embedded directly in the project, the interface document is automatically generated by writing comments during development, and the cost is low	It requires manual input according to the rules of the platform, which is costly	It needs to be entered manually according to the agreed specifications, and the cost is high

If possible, you can determine which platform to use to manage the interface based on the development cost and the difficulty of parsing the interface file

Our project is a mix of Swagger and WIKI, and since most of the daily tests look at WIKIs, we used BeautifulSoup, a Python crawler, to parse WIKI html pages in the early days

from bs4 import BeautifulSoup

soup = BeautifulSoup(html_doc)

title_string = soup.title.string

# Continue to parse other interfaces that need to be used later

Some of the drawbacks of using the wiki to get interface information are discovered

It is unreliable to rely entirely on manual constraints on writing rules
For complex nested parameters, the slightest deviation from the specification will lead to script parsing errors, which greatly affects the difficulty of parsing
Accurate positioning information on HTML is far more difficult than JSON, and the compatibility is poor

Therefore, try to parse the json returned by Swagger to obtain the interface information in preparation for the later generation of the script

"swagger": "2.0",

"host": "xxx",

"basePath": "/",

"tags":[

{"name":"xxx-controller","description":"xxx"},

...

"paths": {

"<接口地址1>": { ... },

"<接口地址2>": { ... },

...

"definitions": {

"<实体类1>": { ... },

"<实体类2>": { ... },

...

}

After using the following methods to get the json result, you can directly obtain the required content in the same way as the dictionary.

graph LR

json-->ApiObj

For swagger.json parsing and code generation, the official also provides some libraries available for use: swagger-codegen (java), due to the limitations of the programming language, we use python to parse it ourselves

Now we have the information we need to generate the code

Automatically generate code

Code generation tools

class CodeGeneratorBackend():

def begin(self, tab="\t"):

self.code = []

self.tab = tab

self.level = 0

def end(self):

# return string.join(self.code, "")

return "".join(self.code)

def write(self, string):

self.code.append(self.tab * self.level + string)

def indent(self):

self.level = self.level + 1

def dedent(self):

if self.level == 0:

raise SyntaxError("internal error in code generator")

self.level = self.level - 1

"""Call the method to start building code""

c = CodeGeneratorBackend()

c.begin(tab=" ") # 定义缩进方式

c.write("def function(self):\n")

c.indent() # 缩进

# Method body

c.dedent() # 回退上一次缩进

Script generation rules for interface classes

Since our interface is stored in the class structure, it can be replaced by traversal according to the API Object interface of the current script

Code generation rules for interface use cases

Special value use cases

Use cases for generating special values like 0, None, and empty strings for each parameter

Targeting parameter type

Based on the type given by the interface parameter, the value that conforms to the type and some values that do not conform to the parameter type (robustness) are generated, and the use case is generated after the value is assigned, as shown in the following code example

Target specific keyword parameters

Pagination test cases can be generated when page-related parameters are encountered, and the details of specific pagination test cases will not be repeated
When encountering parameters such as starttime and endtime, you can generate a use case that compares two time parameters with the current time, and a use case that compares two time parameters before and after

The generation rules need to agree on some basic principles with the development, and we also need to summarize more in the daily testing, find out those use cases with fixed rules, and find ways to locate and generate such use cases

Locate the interface type

Query APIs: Use cases such as single-parameter query, combined parameter query, and full-parameter query can be generated
Update APIs: You can generate use cases such as updating each parameter in a single entry, combining updates, and full updates

Introduction to the Auto-Generate Test Script Tool

Frame flowchart

Tool extensibility

As you can see from the framework diagram, the use case rules can quickly become independent templates and can be maintained separately to facilitate the addition of new rules in the future
The code template is extensible, and different teams have different styles for code specifications and basic templates, and the style of the generated template can be customized, which increases the flexibility of the tool
It supports a variety of data type conversions, and can be extended to generate API objects, parameter dictionaries, or other data schemas

Outcomes and follow-up

Efficiency improvement

Taking a coupon requirement as an example, about 10 APIs (about 150 parameter request parameters, 100 return parameters) have been added/updated, including several types of addition, deletion, modification and query, and the comparison of the time spent before and after using the tool to write and debug scripts, as follows:

type	Description of the workload	No tools are used	Use the tool	Efficiency improvement
Interface classes	Approx. 250 parameters	2 days/person	Within 1 hour	94%
Examples of health and sophistication	Approximately 1,000 use cases	2 days/person	1 day/person	50%
average	---	---	---	74%

As can be seen from the above example, the efficiency increased by nearly half after using the script, and it only took 2~3 working days from design to writing the first version of the tool, which is still very worthwhile.

Spotlight testing

The reduction in scripting effort increases the time spent thinking about product testing, refining use cases, checking coverage, etc

Uniform specifications

Unified interface input and writing specifications, which is convenient for teams to maintain and understand scripts
Unify the idea of basic use case generation to avoid omissions by test engineers when designing basic use case design; Unify the use case output format so that others can understand and maintain the use case

Refactoring tools

If you plan to do script refactoring, you can use the tool to save many times the time of writing interface information and part of the script for the use case

Optimization points for subsequent iterations

At present, most of the generation ideas for use cases are still limited to single parameters, and there are still few generation ideas for multi-parameters, and more generation ideas for use cases will be expanded through brainstorming and other forms in the future
By actually calling APIs, the results are obtained, which improves the accuracy of automatically generating the expected results of the use case, and then saves more time investment in adjusting some of the desired results

Finally, I want to say that the design idea of this gadget is far more important than the implementation, no matter which language or library can be used to generate parse files and code, the important thing is what kind of idea to generate scripts, and we have a lot to explore in this part of the follow-up.

Author: Yuting

Source-WeChat public account: Hujiang Technology

Source: https://mp.weixin.qq.com/s/H1VJ2x6YD9LNytpp4Mtf9A

Analyze the scheme of automatically generating test scripts