天天看點
傳回

Nginx 負載均衡的Cache緩存批量清理的操作記錄

1）nginx.conf配置 

[root@inner-lb01 ~]# cat /data/nginx/conf/nginx.conf
user  www;
worker_processes  8;
 
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;
 
#pid        logs/nginx.pid;
 
 
events {
    worker_connections  65535;
}
 
 
http {
    include       mime.types;
    default_type  application/octet-stream;
    charset utf-8;
       
    ######
    ## set access log format
    ######
    log_format  main  '$remote_addr $remote_user [$time_local] "$request" '
                  '$status $body_bytes_sent "$http_referer" '
                  '$http_user_agent $http_x_forwarded_for $request_time $upstream_response_time $upstream_addr $upstream_status';
 
    #######
    ## http setting
    #######
    sendfile       on;
    tcp_nopush     on;
    tcp_nodelay    on;
    keepalive_timeout  65;
    proxy_cache_path /var/www/cache levels=1:2 keys_zone=mycache:20m max_size=2048m inactive=60m;
    proxy_temp_path /var/www/cache/tmp;
 
    fastcgi_connect_timeout 3000;
    fastcgi_send_timeout 3000;
    fastcgi_read_timeout 3000;
    fastcgi_buffer_size 256k;
    fastcgi_buffers 8 256k;
    fastcgi_busy_buffers_size 256k;
    fastcgi_temp_file_write_size 256k;
    fastcgi_intercept_errors on;
 
    #
    client_header_timeout 600s;
    client_body_timeout 600s;
   # client_max_body_size 50m;
    client_max_body_size 100m;
    client_body_buffer_size 256k;
 
    gzip  on;
    gzip_min_length  1k;
    gzip_buffers     4 16k;
    gzip_http_version 1.1;
    gzip_comp_level 9;
    gzip_types       text/plain application/x-javascript text/css application/xml text/javascript application/x-httpd-php;
    gzip_vary on;
 
    ## includes vhosts
    include vhosts/*.conf;
}

針對上面的配置，注意下面兩點：

1. log_format如上的配置後，nginx的log日志中就能顯示用戶端通路的來源ip了。如下：

[root@inner-lb01 ~]# tail -10 /data/nginx/logs/bs7001.wangshibo.cn-access.log

172.16.51.163 - [21/Nov/2017:17:01:46 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.033 0.033 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:01:46 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.021 0.021 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:01:52 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.028 0.028 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:01:52 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.022 0.022 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:01:58 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.033 0.033 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:01:58 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.021 0.021 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:02:04 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.034 0.034 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:02:04 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.021 0.021 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:02:10 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.031 0.031 172.16.51.161:7001 200
172.16.51.163 - [21/Nov/2017:17:02:10 +0800] "POST /SmartPage/baseBusinessEventServer.service HTTP/1.1" 200 408 "-" Java/1.7.0_79 - 0.022 0.022 172.16.51.161:7001 200

2. nginx的靜态緩存配置：

proxy_cache_path /var/www/cache          表示定義緩存存儲目錄,手動建立；

levels=1:2          緩存級别,表示緩存目錄的第一級目錄是1個字元，第二級目錄是2個字元；

keys_zone=mycache:20m      核心中建立用于緩存緩存資料源資料的空間，查找緩存的時候，先從這個核心空間中找到，緩存資料的源資料，然後再到對應目錄中查找緩存；

max_size=2048m        緩存空間最大值；

inactive=60m         緩存的資料，60分鐘内沒有被通路過就删除

proxy_temp_path /var/www/cache/tmp         表示建立緩存的時候可能生成一些臨時檔案存放的位置，自動建立

2）vhosts虛拟目錄的upstream配置

[root@inner-lb01 ~]# cat /data/nginx/conf/vhosts/bs7001.wangshibo.cn.conf 
upstream bs7001 {
      ip_hash;
      server 172.16.51.161:7001 max_fails=3 fail_timeout=15s;
      server 172.16.51.162:7001 max_fails=3 fail_timeout=15s;
}
          
  server {
      listen      80;
      server_name bs7001.wangshibo.cn;
    
      access_log  /data/nginx/logs/bs7001.wangshibo.cn-access.log main;
      error_log  /data/nginx/logs/bs7001.wangshibo.cn-error.log;
    
 location / {
         proxy_pass http://bs7001;
         proxy_redirect off ;
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header REMOTE-HOST $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
         proxy_connect_timeout 300;
         proxy_send_timeout 300;
         proxy_read_timeout 600;
         proxy_buffer_size 256k;
         proxy_buffers 4 256k;
         proxy_busy_buffers_size 256k;
         proxy_temp_file_write_size 256k;
         proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504 http_404;
         proxy_max_temp_file_size 128m;
         proxy_cache mycache;
         proxy_cache_valid 200 302 1h; 
         proxy_cache_valid 301 1d;
         proxy_cache_valid any 1m;
        }
}

由上面的配置需要注意：

proxy_cache mycache;            表示記憶體緩存源資料空間名字，nginx.conf檔案裡已經定義

proxy_cache_valid 200 302 1h;       表示200、302狀态的頁面資源緩存1小時

proxy_cache_valid 301 1d;        表示301狀态的頁面資源緩存1天

proxy_cache_valid any 1m;       他緩存1分鐘

 3）清理cache緩存

[root@inner-lb01 ~]# ll /var/www/cache/
total 68
drwx------ 27 www www  4096 Nov 21 16:24 0
drwx------ 24 www www  4096 Nov 21 16:24 1
drwx------ 27 www www  4096 Nov 21 16:24 2
drwx------ 21 www www  4096 Nov 21 17:18 3
drwx------ 35 www www  4096 Nov 21 16:24 4
drwx------ 31 www www  4096 Nov 21 16:24 5
drwx------ 34 www www  4096 Nov 21 14:38 6
drwx------ 28 www www  4096 Nov 21 14:38 7
drwx------ 26 www www  4096 Nov 21 16:25 8
drwx------ 23 www www  4096 Nov 21 16:24 9
drwx------ 25 www www  4096 Nov 21 16:13 a
drwx------ 25 www www  4096 Nov 21 14:58 b
drwx------ 21 www www  4096 Nov 21 16:24 c
drwx------ 18 www www  4096 Nov 21 16:12 d
drwx------ 18 www www  4096 Nov 21 16:25 e
drwx------ 20 www www  4096 Nov 21 16:24 f
drwx------  2 www root 4096 Nov 21 17:18 tmp

編寫批量清理緩存的腳本

[root@inner-lb01 ~]# cat /opt/nginx_cache_clean.sh 
#!/bin/bash  
#Email:[email protected]
#Auto Clean Nginx Proxy_Cache Shell Scripts  
#Aunthor:wanghsibo
#Date:2017-11-20 
echo -e "\n\n"  
echo -n -e "\e[35;1m請輸入Nginx Proxy_cache緩存的具體路徑(友情提示:可以使用Tab補全功能哦!)\e[0m\e[34;5m:\e[0m"  
read -e path  
CACHE_DIR=$path  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -n -e "\e[32;1m請輸入你要删除的動作\n1.按檔案類型删除\t2.按具體檔案名删除\t3.按檔案目錄删除\n:"  
read action  
     case $action in  
1)  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -n -e "\e[34;1m 請輸入你要删除的緩存檔案類型(可以輸入多個參數空格隔開)\e[0m\e[34;5m:\e[0m"  
read -a FILE  
for i in `echo ${FILE[*]}|sed 's/ /\n/g'`  
do  
grep -r -a  \.$i ${CACHE_DIR}| awk 'BEGIN {FS=":"} {print $1}'  > /tmp/cache_list.txt  
 for j in `cat /tmp/cache_list.txt`  
do  
   rm  -rf  $j  
   echo "$i  $j 删除成功!"  
 done  
done  
;;  
2)  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -n -e "\e[33;1m 請輸入你要删除的緩存檔案具體名稱(可以輸入多個參數空格隔開)\e[0m\e[34;5m:\e[0m"  
read -a FILE  
for i in `echo ${FILE[*]}|sed 's/ /\n/g'`  
do  
grep -r -a  $i ${CACHE_DIR}| awk 'BEGIN {FS=":"} {print $1}'  > /tmp/cache_list.txt  
 for j in `cat /tmp/cache_list.txt`  
do  
   rm  -rf  $j  
   echo "$i  $j 删除成功!"  
 done  
done  
;;  
3)  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -e "\e[32;1m----------------------------------------------------------------\e[0m"  
echo -n -e "\e[33;1m支援的模式有:\n1.清除網站store目錄下的所有緩存:test.dd.com/data/upload/shop/store\n2.清除網站shop下的所有緩存:test.dd.com/data/upload/shop\e[0m\n"  
echo -n -e "\e[34;1m 請輸入你要删除的緩存檔案具體目錄\e[0m\e[34;5m:\e[0m"  
read -a FILE  
for i in `echo ${FILE[*]}|sed 's/ /\n/g'`  
do  
grep -r -a  "$i" ${CACHE_DIR}| awk 'BEGIN {FS=":"} {print $1}'  > /tmp/cache_list.txt  
 for j in `cat /tmp/cache_list.txt`  
do  
   rm  -rf  $j  
   echo "$i  $j 删除成功!"  
 done  
done  
;;  
*)  
echo "輸入錯誤,請重新輸入"  
;;  
esac

執行腳本：

[root@inner-lb01 ~]# chmod 755 /opt/nginx_cache_clean.sh
[root@inner-lb01 ~]# sh /opt/nginx_cache_clean.sh



請輸入Nginx Proxy_cache緩存的具體路徑(友情提示:可以使用Tab補全功能哦!):/var/www/cache/
----------------------------------------------------------------
----------------------------------------------------------------
請輸入你要删除的動作
1.按檔案類型删除   2.按具體檔案名删除  3.按檔案目錄删除
:1
----------------------------------------------------------------
----------------------------------------------------------------
 請輸入你要删除的緩存檔案類型(可以輸入多個參數空格隔開):html htm js css jpg gif png jpeg bmp flv swf ico
html  /var/www/cache/6/5e/75685391e510dde1671cb198c113e5e6 删除成功!
html  /var/www/cache/3/b7/49e71a93813bcedd9e0c679bc20c1b73 删除成功!
html  /var/www/cache/3/b7/49e71a93813bcedd9e0c679bc20c1b73 删除成功!
html  /var/www/cache/8/86/ee4a14c4cc9e12503f9e8bef123df868 删除成功!
html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
html  /var/www/cache/8/53/a9e2ef17aba2c92aac71b36a50329538 删除成功!
js  /var/www/cache/f/3c/6a74478ecb76c1235ef47766e1b333cf 删除成功!
js  /var/www/cache/f/3c/6a74478ecb76c1235ef47766e1b333cf 删除成功!
js  /var/www/cache/1/2b/eeabd84e3ed8cc4bec61f608e2ab32b1 删除成功!
js  /var/www/cache/6/98/e38773f35e9e18fe4449396105faf986 删除成功!
js  /var/www/cache/e/d6/5c7f20342601e4cee752424175e35d6e 删除成功!
js  /var/www/cache/8/f9/ec9fa02ace81a82d98baa04b3d810f98 删除成功!
js  /var/www/cache/7/62/4ad7e6a1ea04cf484a911250e5af5627 删除成功!
js  /var/www/cache/7/62/4ad7e6a1ea04cf484a911250e5af5627 删除成功!
js  /var/www/cache/c/a7/1edf94c22612d6e7f2cb2c0b289dba7c 删除成功!
js  /var/www/cache/a/2e/6673665a2ec9fae7640f61c121ae42ea 删除成功!
css  /var/www/cache/7/45/26ddc4b8c5085be9757b90cd4a36a457 删除成功!
css  /var/www/cache/7/45/26ddc4b8c5085be9757b90cd4a36a457 删除成功!
css  /var/www/cache/7/7f/cb39e58e71c9fb5a1ba77107288167f7 删除成功!
css  /var/www/cache/7/7f/cb39e58e71c9fb5a1ba77107288167f7 删除成功!
css  /var/www/cache/a/45/9270775ac808bde171eda5f39547845a 删除成功!
css  /var/www/cache/a/45/9270775ac808bde171eda5f39547845a 删除成功!
css  /var/www/cache/a/45/9270775ac808bde171eda5f39547845a 删除成功!
css  /var/www/cache/2/40/ab83b9cdeec1955375d28f6e74b6e402 删除成功!
css  /var/www/cache/2/40/ab83b9cdeec1955375d28f6e74b6e402 删除成功!
png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
png  /var/www/cache/f/a0/eaa602c9070ef04101a5c148d63b4a0f 删除成功!
png  /var/www/cache/1/f7/3ba2c5ee13c4a086ed885221cb1a9f71 删除成功!
png  /var/www/cache/1/f7/3ba2c5ee13c4a086ed885221cb1a9f71 删除成功!
png  /var/www/cache/6/f3/5e00a9011498e811a075031373928f36 删除成功!
png  /var/www/cache/6/f3/5e00a9011498e811a075031373928f36 删除成功!
png  /var/www/cache/8/70/03a64ee54acbe99ce81d2e907edec708 删除成功!

上面腳本執行後，會提示輸入cache的緩存目錄，然後選擇删除緩存檔案的條件（這裡我選擇"按檔案類型删除"），選擇了删除html 、htm、js、css、jpg 、gif、 png 、jpeg 、bmp 、flv、 swf 、ico這12中檔案格式的緩存檔案。

或者直接使用find指令查找緩存目錄下的檔案，直接将檔案全部删除

# find /var/www/cache -type f|xargs rm -f

-------------------Nginx禁用html、js、css緩存-----------------

在本地開發的時候，經常會碰到緩存引起的莫名其妙的問題，最暴力的方式就是清掉浏覽器的緩存，或者使用Ctrl + F5，Shift + F5強制重新整理頁面。
有時候按了好幾下，緩存還是清不掉，隻能暫時禁用浏覽器靜态資源緩存了，配置如下：
location ~.*\.(js|css|html|png|jpg)$
{
    add_header Cache-Control no-cache;
}

或者
location /js
{
    add_header Cache-Control no-cache;
}

現在，按F5就行了！

*************** 當你發現自己的才華撐不起野心時，就請安靜下來學習吧！***************

上一篇: 從張勇對話諾貝爾經濟學獎獲得者，看阿裡平台的本質
下一篇: 雲原生時代，CNStack 如何解決企業數字化轉型難題？