1. Overview
Rate limiting is a form of service degradation that protects the system by limiting the input and output traffic of the system.
For example, if our website is exposed to the public network environment, in addition to the normal access of users, sudden traffic such as web crawlers, malicious attacks or big promotions may cause pressure on the system, and if this pressure exceeds the processing capacity of the server, it will cause the problem of slow response or even system crash.
Therefore, when the number of concurrent requests is too large, we ensure that the server can correctly respond to the other part of the request by limiting one part of the request (such as limiting frequent requests from the same IP).
nginx provides two ways to throttle traffic, one is to limit the request rate and the other is to limit the number of connections.
Limits on download/upload speeds are also provided.
2. Limit the request rate
NGINX's ngx_http_limit_req_module module provides the ability to throttle the rate at which requests are processed, using the leaky bucket algorithm. If there is water in the bucket, the water that just goes in will be stored in the bucket and will not flow out after the water below is finished, and if the speed of the water inlet is greater than the speed of the water flowing out, the water in the bucket will be full, and the water will not enter the bucket, but overflow directly from the top of the bucket.
Corresponding to the processing of network requests, water represents requests from clients, while buckets represent a queue in which requests are waiting to be processed according to a first-in, first-out (FIFO) algorithm. A leak represents a request leaving the buffer and being processed by the server, and an overflow represents a request being dropped and never served.
2.1, Normal current limit
There are two main directives in nginx that can be used to configure throttling: limit_req_zone and limit_req.
Here's an example of the simplest throttling:
limit_req_zone $binary_remote_addr zone=test:10m rate=2r/s;
server {
location / {
limit_req zone=test;
}
}
imit_req_zone Parameters used to set the current limit and shared memory zone in the following format: limit_req_zone key zone rate.
- key: Defines the throttling object, $binary_remote_addr is a variable in nginx, which means that the throttling is based on the remote_addr (client IP), and binary_ is binary storage. $binary_remote_addr is used instead of $remote_addr because binary storage can compress the memory footprint. The size of the $remote_addr variable varies from 7 to 15 bytes, while the size of the $binary_remote_addr variable is always 4 bytes for IPv4 and 16 bytes for IPv6 addresses.
- zone: Defines a shared memory zone to store access information, including the status of each IP address and the frequency of access restricted request URLs. The definition of zone is further divided into two parts: the name of the zone, identified by the zone= keyword, and the size of the zone after the colon. test:10m indicates a memory area with a size of 10M and the name test. 1M can store access information for 16,000 IP addresses, and test can store about 160,000 addresses. When nginx creates a new record, it will remove the records that have not been used in the previous 60 seconds, and if the freed space still cannot store new records, it will return a status code of 503.
- rate: Sets the maximum access rate. rate=2r/s (for good simulation, the value of rate is set to be small), which means that a maximum of 2 requests per second can be processed. In fact, nginx tracks requests at a millisecond granularity, rate=2r/s is actually 1 request every 500 milliseconds, that is, after the previous request is completed, if there are still requests arriving within 500 milliseconds, these requests will be rejected (503 is returned by default, and if you want to modify the return value, you can set the limit_req_status).
limit_req_zone just sets the throttling parameters, which must be used with limit_req if they are to take effect. The format of the limit_req is: limit_req zone=name [burst=number] [nodelay].
The above example simply specifies zone=test, which means that the configuration of the test area is used to throttle the HTML file when requesting it. We can understand that this bucket does not currently have any capacity to store water droplets, and all requests that arrive that cannot be leaked immediately will be denied. If I send 10 requests in 1 second, with 1 request for the first 500 ms and 9 requests for the last 500 ms, then only the first request for the first 500 ms and the first request for the last 500 ms will be responded, and the rest of the requests will be rejected.
2.2. Handle burst traffic
The above configuration guarantees that nginx will serve at a fixed speed (2r/s), but this situation does not apply to bursts of traffic, and we want to cache requests and process them as much as possible, in which case we need to add a burst parameter to the limit_req:
limit_req_zone $binary_remote_addr zone=test:10m rate=2r/s;
server {
location / {
limit_req zone=test burst=5;
}
}
burst represents the number of additional requests that can be processed after the set access rate is exceeded. When rate=2r/s, it means that one request can be processed every 500ms. burst=5, if 10 requests arrive at the same time, nginx will process the first request, 5 of the remaining 9 requests will be put into the queue, and the remaining 4 requests will be rejected directly. Then get a request from the queue every 500ms for processing, at this time, if there are more requests coming in, if the number of requests in the queue exceeds 5, it will be rejected, and if it is less than 5, it will be added to the queue for waiting. We can understand that the current bucket can hold 5 drops of water:
After the burst is configured, although all requests that arrive at the same time will not be rejected, they still need to wait for 500 ms for the processing time, and the fifth request in the bucket needs to wait for 500 ms * 4 time to be processed. In this case, we need to add the nodelay parameter to use it with burst.
limit_req_zone $binary_remote_addr zone=test:10m rate=2r/s;
server {
location / {
limit_req zone=test burst=5 nodelay;
}
}
nodelay indicates no delay. When nodelay is set, the first request to arrive and the request in the queue are processed immediately, and there are no waiting requests.
It should be noted that although the 5 requests in the queue were processed immediately, the positions in the queue were still released at a rate of 500ms. The next 4 requests are still rejected, and in the long run, the upper limit of throughput will not be increased, and the upper limit of long-term throughput is determined by the set rate.
2.3. Set up a whitelist
If you do not need to limit the rate, such as the test to be stress-tested, you can configure a whitelist to cancel the rate limiting setting. The whitelist uses nginx's ngx_http_geo_module and ngx_http_map_module modules.
geo $limit {
default 1;
10.0.0.0/8 0;
192.168.0.0/24 0;
}
map $limit $limit_key {
0 "";
1 $binary_remote_addr;
}
limit_req_zone $limit_key zone=mylimit:10m rate=2r/s;
The geo directive can create a variable $limit based on the IP. The default value for $limit is 1, and if the following IP is matched, the corresponding value is returned (0 is returned here).
Then, use the map command to map the value of $limit to $limit_key: if it is in the whitelist, the $limit_key is an empty string, and if it is not in the whitelist, it is $binary_remote_addr. When the first parameter of the limit_req_zone directive is an empty string, the throttling does not work, so the whitelisted IP addresses (in the 10.0.0.0/8 and 192.168.0.0/24 subnets) are not restricted, and all other IP addresses are limited to 2r/s
2.4. limit_req repetition
If multiple limit_req directives are configured under the same location, the limits defined by those directives will be used.
geo $limit {
default 1;
10.0.0.0/8 0;
192.168.0.0/24 0;
}
map $limit $limit_key {
0 "";
1 $binary_remote_addr;
}
limit_req_zone $limit_key zone=mylimit:10m rate=2r/s;
limit_req_zone $binary_remote_addr zone=myLimit2:10m rate=10r/s;
server {
location ~* \.(html)$ {
limit_req zone=mylimit burst=5 nodelay;
limit_req zone=myLimit2 burst=5 nodelay;
}
}
The above example is configured with two rules, myLimit and myLimit2. Although the whitelist users do not match the mylimit rule, they are limited to 10r/s according to the mylimit2 rule. For users who are not on the whitelist, you need to match both mylimit and mylimit2, and the most stringent condition of the two, 2r/s, will work.
3. Limit the number of connections
NGINX's ngx_http_limit_conn_module module provides the ability to limit the number of connections, consisting of two directives limit_conn_zone and limit_conn, in the format of limit_conn_zone key zone.
limit_conn_zone $binary_remote_addr zone=perip:10m;
limit_conn_zone $server_name zone=perserver:10m;
server {
location ~* \.(html)$ {
limit_conn perip 10;
limit_conn perserver 100;
}
}
- limit_conn perip 10: The key is $binary_remote_addr, which means that a single IP address can hold a maximum of 10 connections at a time.
- limit_conn perserver 100: The key is $server_name, which means that the total number of concurrent connections that can be handled by the virtual host (server) at the same time is 100.
It should be noted that the connection is counted only when the request header is processed by the backend server.
4. Upload/download rate limit
limit_rate is mainly used to limit the number of bytes transferred between the user and the server, and the most common scenario is probably download/upload rate limiting. limit_rate does not have a separate module, but is in the ngx_http_core_module, and at the same time, it has relatively few related instructions, only two instructions, limit_rate and limit_rate_after.
4.1、limit_rate
server {
location / {
limit_rate 4k;
}
}
- The usage of the limit_rate is very simple, and the rate that follows is the threshold of the specific speed limit
- Note that the default unit is bytes/s, that is, the number of bytes transferred per second Bytes instead of bits
- Rate can be set as a variable, allowing for dynamic rate limiting
- For example, the rate limit is limited to 4K per connection, that is, when the client initiates two connections, the rate can be changed to 8K
4.2、limit_rate_after
server {
location / {
limit_rate_after 500k;
limit_rate 4k;
}
}
limit_rate_after allows rate limiting after a portion of data has been transferred, for example, in the above configuration, there is no rate limit for the first 500k of data transferred, and the rate is limited after 500k. Common application scenarios include segmented download speed limiting, which limits the speed of the part that exceeds the specified size, or streaming video websites generally do not limit the speed of the first screen to ensure the user experience, to ensure that it can be loaded as soon as possible, and then limit the bandwidth to a reasonable range after the user starts watching the video, so as to reduce the additional cost caused by the excessive content loaded in advance due to the fast network speed of the client.
4.3、proxy_limit_rate
The basic principle and usage of the proxy_limit_rate is almost the same as limit_rate, the only difference is that proxy_limit_rate limits the connection rate between nginx and the backend upstream server, while limit_rate limits the connection rate between nginx and the client. It should be noted that proxy_limit_rate need to be turned on proxy_buffering this order will take effect.
#语法：
Syntax: proxy_limit_rate rate;
Default: proxy_limit_rate 0;
Context: http, server, location
This directive appeared in version 1.7.7.
4.4. Dynamic speed limiting
One of the major features of limit_rate is the ability to use variables, which means that dynamic rate limiting can be achieved in combination with map commands and the like, and here are just a few simple examples
4.4.1. Time-based dynamic speed limiting
This introduces a built-in SSI module of NGINX, which has two interesting time variables: $date_local and $date_gmt, which correspond to the current time and GMT time, respectively
Here, we use a combination of variables and map instructions, use regular expressions to match different time periods, and then combine map variables to match different time periods and different speed limits.
map $date_local $limit_rate_time {
default 4K;
~(00:|01:|02:|03:|04:|05:|06:|07:).*:.* 16K;
~(08:|12:|13:|18:).*:.* 8K;
~(19:|20:|21:|22:|23:).*:.* 16K;
}
limit_rate $limit_rate_time
4.2. Dynamic speed limit based on variables
Some services may have different speed limits for users who do not use them, for example, VIP users are faster, etc., for example, you can limit the speed for different cookies below
map $cookie_User $limit_rate_cookie {
gold 64K;
silver 32K;
copper 16K;
iron 8K;
}
limit_rate $limit_rate_cookie
Source: blog.csdn.net/cold___play/article/details/132094865
That's all for today's sharing, if it helps, welcome to support it with one click triple (like, comment, forward)!
Reader-only group: We sincerely invite you to join the technical exchange group and roll together!
If there are any errors or other problems, please leave comments and corrections. If it helps, welcome to like + forward and share. For more related open source technical articles, please stay tuned!Resource sharing (Xiaobian has carefully prepared various academic Xi materials for 2048G.) Including system operation and maintenance, database, redis, MogoDB, e-book, Java basic course, Java practical project, architect comprehensive tutorial, architect practical project, big data, Docker container, ELK Stack, machine learning Xi, BAT interview intensive lecture video, etc. ）