天天看點
傳回

測試常用curl工具詳解前言

前言

linux中測試網絡的通路工具 
title:      CDN中測試常用curl工具詳解
author:     sheazhang

複制

1. 驗證某節點ip緩存情況 

<p style="line-height: 2em;">[root@VM_27_201_centos ~]# curl -I 'http://app613.imgcache.xxx.com/app613/frontend_swf/version.js' -x 106.74.23.144:80
HTTP/1.1 200 OK
Server: NWS_TCloud_S1
Connection: keep-alive
Date: Sun, 03 Jul 2016 08:50:34 GMT
Cache-Control: max-age=6000
Expires: Sun, 03 Jul 2016 10:30:34 GMT
Last-Modified: Thu, 30 Jun 2016 09:12:30 GMT
Content-Type: application/x-javascript; charset=utf-8
Content-Length: 69
X-Cache-Lookup: Hit From Disktank<br></p>

複制

PS:

其中的-x綁定代理伺服器ip的用法，類似于wget中的：

# wget -e http-proxy=119.97.137.178 http://sdxxx.cloudcdn.net/test008/6wireshark-win64-1.2.9.exe

複制

get back

2. 驗證referer頭部白名單是否生效

場景

使用-H 來添加自定義的頭部，模拟特殊條件下源站/cdn節點對特性是否支援，結果傳回是否符合預期。下面的這個例子，就是用來驗證在控制台上添加了referer白名單後，模拟該referer頭部傳回是否符合預期。

<p style="line-height: 2em;">[root@VM_27_201_centos ~]# curl -I 'http://res2.xxxxx.com/' -x 106.74.23.144:80 -H"referer:http://*.yqh5.cn"
HTTP/1.1 200 OK
Server: NWS_TCloud_S1
Connection: keep-alive
Date: Sun, 03 Jul 2016 08:59:50 GMT
Cache-Control: max-age=31536000
Expires: Mon, 03 Jul 2017 08:59:50 GMT
Last-Modified: Tue, 12 Apr 2016 01:59:32 GMT
Content-Type: text/html
Content-Length: 612
X-Cache-Lookup: Hit From Disktank
Access-Control-Allow-Origin: *
Content-Disposition: inline; filename=""
Accept-Ranges: bytes<br></p>

複制

PS 上面的

curl -I 'http://res2.eqxxx.com/' -x 106.74.23.144:80 -H"referer:http://*.yqh5.cn"

複制

也可以寫為：

curl -I 'http://res2.exxx.com/' -x 106.74.23.144:80  -e "http://pachong.org/"

複制

get back

3. 驗證節點/源站對分片的支援情況 

[root@VM_27_201_centos ~]# curl -Iv 'http://update01.xxxx-cloud.com/pup/8S70_E6200_V016.002.230_9.zip' -x 106.74.23.144:80 -H'range:bytes=0-1000'
* About to connect() to proxy 106.74.23.144 port 80 (#0)
*   Trying 106.74.23.144... connected
* Connected to 106.74.23.144 (106.74.23.144) port 80 (#0)
> HEAD http://update01.skyworth-cloud.com/pup/8S70_E6200_V016.002.230_9.zip HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: update01.skyworth-cloud.com
> Accept: */*
> Proxy-Connection: Keep-Alive
> range:bytes=0-1000
>
< HTTP/1.1 206 Partial Content
HTTP/1.1 206 Partial Content
< Server: nws_ocmid_hy
Server: nws_ocmid_hy
< Connection: close
Connection: close
< Date: Sun, 03 Jul 2016 09:02:24 GMT
Date: Sun, 03 Jul 2016 09:02:24 GMT
< Cache-Control: max-age=600
Cache-Control: max-age=600
< Expires: Sun, 03 Jul 2016 09:12:24 GMT
Expires: Sun, 03 Jul 2016 09:12:24 GMT
< Last-Modified: Mon, 23 May 2016 20:10:06 GMT
Last-Modified: Mon, 23 May 2016 20:10:06 GMT
< Content-Range: bytes 0-1000/437405243
Content-Range: bytes 0-1000/437405243
< Content-Type: application/octet-stream
Content-Type: application/octet-stream
< Content-Length: 1001
Content-Length: 1001
< X-Daa-Tunnel: hop_count=2
X-Daa-Tunnel: hop_count=2
< X-Cache-Lookup: Hit From Disktank Upstream
X-Cache-Lookup: Hit From Disktank Upstream
< X-Cache-Lookup: Hit From Disktank
X-Cache-Lookup: Hit From Disktank
< X-Daa-Tunnel: hop_count=2
X-Daa-Tunnel: hop_count=2
< X-Cache-Lookup: Hit From Inner Cluster
X-Cache-Lookup: Hit From Inner Cluster
 
<
* Closing connection #0<br><br></p>

複制

備注

上面的

curl -Iv 'http://update01.xxx-cloud.com/pup/8S70_E6200_V016.002.230_9.zip' -x 106.74.23.144:80 -H'range:bytes=0-1000'

複制

也可以寫成

curl -Iv 'http://update01.xxx-cloud.com/pup/8S70_E6200_V016.002.230_9.zip' -x 106.74.23.144:80 -r 0-1000

複制

get back

4. 驗證檔案在節點和源站上大小是否一緻（通過md5判斷） 

[root@VM_27_201_centos ~]# curl -o pic1 'http://as.xxx.com/show/visitor/4a2d8aae508506ef0150a6fcebbf2f53' -x 106.74.23.144:80
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10126  100 10126    0     0  40629      0 --:--:-- --:--:-- --:--:-- 54149
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]# md5sum pic1
4b7671dbe1a634541726bd9473d14dcc  pic1
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]# curl -o pic2 'http://as.xxx.com/show/visitor/4a2d8aae508506ef0150a6fcebbf2f53' -x 115.159.20.55:80
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10126  100 10126    0     0  4629k      0 --:--:-- --:--:-- --:--:-- 9888k
[root@VM_27_201_centos ~]#
[root@VM_27_201_centos ~]# md5sum pic2
4b7671dbe1a634541726bd9473d14dcc  pic2<br></p>

複制

備注

上面的-o 代表把檔案下載下傳到指定的路徑下，-x代表使用指定的ip或者域名作為代理。

get back

5. 模拟https通路驗證https證書 

[root@VM_117_214_centos ~]#  curl -sv 'https://xxx.qcloud.com/static/img/57c45bb3997a277e5996dcf13314b365/user-40.png' -o /dev/null -k
* About to connect() to mccdn.qcloud.com port 443 (#0)
*   Trying 113.207.16.63... connected
* Connected to mccdn.qcloud.com (113.207.16.63) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* warning: ignoring value of ssl.verifyhost
* skipping SSL peer certificate verification
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*     subject: CN=*.qcloud.com,OU=R&D,O=Shenzhen Tencent Computer Systems Company Limited,L=shenzhen,ST=guangdong,C=CN
*     start date: May 25 00:00:00 2016 GMT
*     expire date: Jul 24 23:59:59 2019 GMT
*     common name: *.qcloud.com
*     issuer: CN=Symantec Class 3 Secure Server CA - G4,OU=Symantec Trust Network,O=Symantec Corporation,C=US
> GET /static/img/57c45bb3997a277e5996dcf13314b365/user-40.png HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: mccdn.qcloud.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nnws/1.7.3.8
< Date: Sat, 20 Aug 2016 12:43:55 GMT
< Content-Type: image/png
< Content-Length: 3558
< Connection: keep-alive
< Cache-Control: max-age=2592000
< Expires: Mon, 19 Sep 2016 12:43:54 GMT
< Last-Modified: Wed, 25 May 2016 08:06:29 GMT
< X-Cache-Lookup: Hit From Disktank
< 
{ [data not shown]
* Connection #0 to host mccdn.qcloud.com left intact
* Closing connection #0

複制

get back

6. curl中綁定host測試方法

等同于在windows中綁定host進行測試

[root@VM_230113centos ~]# curl -I -H 'host:7u2q8y.com3.z0.glb.xxxcdn.com' http://xxx.com.z0.glb.xxxdns.com/M-_WoLgVqduR7VFfrhnmWwL8YgA=/luI-
 
lideKDodDAYi55H6D2HV6HcO/000000.ts
HTTP/1.1 200 OK
Server: nginx/1.4.4
Date: Sat, 02 Jan 2016 11:39:23 GMT
Content-Type: video/mp2t
Content-Length: 673416
Connection: keep-alive
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 2592000
Cache-Control: public, max-age=31536000
Content-Disposition: inline; filename="000000.ts"
Content-Transfer-Encoding: binary
Etag: "Fs–ekImTQ2ZMjO_MdTeTN0Yew_5"
Last-Modified: Sun, 22 Nov 2015 06:28:55 GMT
X-Log: mc.g;IO:1
X-Reqid: iToAAMIdW1cTmSUU
X-Qiniu-Zone: 0

[root@VM_230113centos ~]# curl -I -H 'host:7u2q8y.com3.z0.glb.xxxcdn.com' http://211.161.127.147/M-_WoLgVqduR7VFfrhnmWwL8YgA=/luI-lideKDodDAYi55H6D2HV6HcO/000000.ts
HTTP/1.1 200 OK
Server: NWS_Appimg_HY
Connection: keep-alive
Date: Sat, 02 Jan 2016 11:45:50 GMT
Cache-Control: max-age=31536000
Expires: Sun, 01 Jan 2017 11:45:50 GMT
Last-Modified: Sun, 22 Nov 2015 06:28:55 GMT
Content-Type: video/mp2t
Content-Length: 673416
X-Cache-Lookup: Hit From Disktank
Access-Control-Allow-Origin: *
Accept-Ranges: bytes
Etag: "Fs–ekImTQ2ZMjO_MdTeTN0Yew_5"
X-ReqId: ATUAABGu8oBVjyUU
Content-Disposition: inline; filename="000000.ts"

複制

其中：

  • 7u2q8y.com3.z0.glb.qiniucdn.com 為使用者的域名
  • src.com.z0.glb.qiniudns.com 為源站或者回源host位址，也可以為ip位址形式
  • 211.161.127.147 為域名接入cdn之後解析到的cdn節點ip位址

通過驗證源站url和cdn節點ip位址後的curl回報可以判斷出到底是源站有問題，還是cdn節點有問題

get back

7. 驗證TCP握手和ssl握手的延時 

curl -w "TCP handshake: %{time_connect}, SSL handshake: %{time_appconnect}\n" -so /dev/null https://www.alipay.com

複制

8. 使用curl抓取傳回時間

判斷網頁通路各耗時時間

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" http://www.itts-union.com
This command will output this:
[root@VM_117_214_centos ~]#  curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} \n" http://www.itts-union.com/1326.html
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 74470    0 74470    0     0  52368      0 --:--:--  0:00:01 --:--:--  116k
Connect: 0.799 TTFB: 1.309 Total time: 1.422

複制

get back

9. 查詢各個階段的耗時時間：

-w 可以指定特殊檔案 也可以指定耗時**%{time_namelookup}**

[root@VM_49_231_centos ~]# curl -Iv "http://img.yxp01.com/
images/20/2017/07/VXunsmRXXU355lC5lM8xc5sXmFNfus.jpg?v=4" -x 
119.6.229.37:80 -so /dev/null -w "@curl-time"

複制

特殊檔案：

[root@VM_49_231_centos ~]# cat curl-time 
    Time-DNS-lookup:  %{time_namelookup}s\n
   Time-TCP-connect:  %{time_connect}s\n
   Time-SSL-connect:  %{time_appconnect}s\n
      Time-redirect:  %{time_redirect}s\n
  Time-Pre-transfer:  %{time_pretransfer}s\n
Time-First-transfer:  %{time_starttransfer}s\n
                    ----------\n
         Time-total:  %{time_total}s\n

複制

get back

10. -L實作301或者302的跳轉

測試常用curl工具詳解前言

11. 緩存檔案建立時間

Pragma: X-Get-Last-Update-Info

測試常用curl工具詳解前言

get back

12. 檢視cdn節點回源的鍊路

x_upstream_info=on

curl -o 100 "http://res.hoisin.xxxtv.com/video/20150731164143_960.ts?x_upstream_info=on" -x 61.140.13.201:80 -H "Pragma: X-Get-Last-Update-Info" -v

[root@sheazhang ~]# curl -o /dev/null -v "http://zl.winyp.cn/test.jpg?x_upstream_info=on"
> GET /test.jpg?x_upstream_info=on HTTP/1.1
> User-Agent: curl/7.29.0
> Host: zl.winyp.cn
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: NWS_TCloud_S1
< Connection: keep-alive
< Date: Tue, 01 Jan 2019 02:46:29 GMT
< Cache-Control: max-age=600
< Expires: Tue, 01 Jan 2019 02:56:29 GMT
< Last-Modified: Thu, 22 Jun 2017 18:27:09 GMT
< Content-Type: image/jpeg
< Content-Length: 201557
< X-NWS-LOG-UUID: 4243021142118093756 4c4df2ad367f7a7282b8c634e5bb5346
< Access-Control-Allow-Origin: *
< X-Daa-Tunnel: hop_count=3
< X-Cache-Lookup: Hit From Upstream 125.39.46.83, cost 33 ms
< X-Upstream-Info: l2.cloudhy.tcdn.qq.com
< X-Cache-Lookup: Hit From Disktank3
< Accept-Ranges: bytes
< X-Cache-Lookup: Hit From Inner Cluster

複制

13. 指定通路的來源用戶端ua

-A 'ua'

[root@sheaBJ ~]# curl -Iv "http://www.xxx.com/ckkhj/index.shtml" -x 1.xxx:80 -A 'Smartphone'
* About to connect() to proxy 1.180.204.168 port 80 (#0)
*   Trying 1.180.204.168...
* Connected to 1.180.204.168 (1.180.204.168) port 80 (#0)
> HEAD http://www.dongao.com/ckkhj/index.shtml HTTP/1.1
> User-Agent: Smartphone
> Host: www.dongao.com
> Accept: */*
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 301 Moved Permanently
HTTP/1.1 301 Moved Permanently
< Server: NWS_TCloud_S1
Server: NWS_TCloud_S1
< Connection: keep-alive
Connection: keep-alive
< Date: Fri, 08 Mar 2019 03:51:38 GMT
Date: Fri, 08 Mar 2019 03:51:38 GMT
< Content-Length: 39
Content-Length: 39
< Location: http://m.dongao.com/ckkhj/index.shtml
Location: http://m.dongao.com/ckkhj/index.shtml

< 
* Excess found in a non pipelined read: excess = 39 url = /ckkhj/index.shtml (zero-length body)
* Connection #0 to host 1.180.204.168 left intact

複制

14. curl常用頭部集合 

-H 'Accept-Encoding:gzip'   測試壓縮
 -L                 實作301或者302的跳轉
 -H 'Pragma: X-Get-Last-Update-Info'     緩存檔案建立時間
 -H 'x_upstream_info=on'			cdn節點回源的鍊路
 -H 'host:7u2q8y.com3.z0.glb.qiniucdn.com'   綁定hosts
 -H 'range:bytes=0-1000'  或 -r 0-0   測試分片支援
 -H "referer:http://*.yqh5.cn"     驗證reffer頭部
 -d "x=123"  	模拟post請求
 -H "Origin:http://sheazhang.winyp.cn/"  驗證跨域的來源情況，其中O大寫
 -w "@curl-time"				各個階段的耗時時間

複制

特殊檔案：

[root@VM_49_231_centos ~]# cat curl-time 
    Time-DNS-lookup:  %{time_namelookup}s\n
   Time-TCP-connect:  %{time_connect}s\n
   Time-SSL-connect:  %{time_appconnect}s\n
      Time-redirect:  %{time_redirect}s\n
  Time-Pre-transfer:  %{time_pretransfer}s\n
Time-First-transfer:  %{time_starttransfer}s\n
                    ----------\n
         Time-total:  %{time_total}s\n

複制

15. httpstat的腳本-time

參考git連結

16. 測試源站是否吐出304的情況

-H 'If-Modified-Since:xxx'

[root@VM_14_42_centos ~]# curl -o 1 -v "http://j.xxx.cc/daobo/js/fabric.js" -x 139.xxx.204:80 -H 'If-Modified-Since: Mon, 23 Apr 2018 03:38:55 GMT'
* About to connect() to proxy 139.199.214.204 port 80 (#0)
*   Trying 139.xxx...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
				 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* Connected to 139.199.214.204 (139.199.214.204) port 80 (#0)
> GET http://j.xxx.cc/daobo/js/fabric.js HTTP/1.1
> User-Agent: curl/7.29.0
> Host: xxx
> Accept: */*
> Proxy-Connection: Keep-Alive
> If-Modified-Since: Mon, 23 Apr 2018 03:38:55 GMT
>
< HTTP/1.1 304 Not Modified
< Server: nginx/1.14.2
< Date: Mon, 15 Apr 2019 07:40:50 GMT
< Connection: keep-alive
< Last-Modified: Mon, 23 Apr 2018 03:38:55 GMT
< ETag: "5add554f-da835"
< Access-Control-Allow-Origin: *
<
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host 139.199.214.204 left intact

複制

17. 其他的不常見用法，僅做了解

1、抓取頁面内容到一個檔案中

[[email protected]]# curl -o home.html  http://www.itts-union.com

複制

2、模拟表單資訊，模拟登入，儲存cookie資訊

[[email protected]]# curl -c ./cookie_c.txt -F log=aaaa -F pwd=****** http://www.itts-union.com/wp-login.php

複制

3、使用cookie檔案

[[email protected]]# curl -b ./cookie_c.txt  http://www.itts-union.com/wp-admin

複制

使用cookie頭部：

[[email protected]]# curl -o /dev/null -v http://www.itts-union.com/wp-admin -H "Cookie: uid=999;sex=2"

複制

4、顯示下載下傳進度條

[root@VM_117_214_centos ~]# curl -o smokeping.tar.gz http://oss.oetiker.ch/smokeping/pub/smokeping-2.6.9.tar.gz -#
######################################################################## 100.0%

複制

5、通過ftp下載下傳檔案

[[email protected] ~]$ curl -u 使用者名:密碼 -O http://www.itts-union.com/demo/curtain/bbstudy_files/style.css
　　% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
　　Dload  Upload   Total   Spent    Left  Speed
　　101  1934  101  1934    0     0   3184      0 –:–:– –:–:– –:–:–  7136

複制

　　 或者用下面的方式

[[email protected] ~]$ curl -O ftp://使用者名:密碼@ip:port/demo/curtain/bbstudy_files/style.css
[[email protected] ~]$ curl -O ftp://使用者名:密碼@ip:port/demo/curtain/bbstudy_files/style.css

複制

6、通過ftp上傳

[[email protected] ~]$ curl -T test.sql ftp://使用者名:密碼@ip:port/demo/curtain/bbstudy_files/

複制

7、如下載下傳網頁發現亂碼，可以使用iconv轉碼：

[root@VM_117_214_centos
 ~]# curl -o smokeping.tar.gz 
http://oss.oetiker.ch/smokeping/pub/smokeping-2.6.9.tar.gz -# | iconv 
-fgb2312
######################################################################## 100.0%

複制

8、驗證https通路

curl -sv 'https://mccdn.qcloud.com/static/img/57c45bb3997a277e5996dcf13314b365/user-40.png' -o/dev/null --resolve "mccdn.qcloud.com:443:221.204.60.11"

複制

get back

上一篇: 移動前端開發和web前端開發的差別
下一篇: spring mvc前端頁面中文亂碼問題解決思路