測試必會之 Linux 三劍客之 grep

grep (global search regular expression(RE) and print out the line,全面搜尋正規表達式并把行列印出來)是一種強大的文本搜尋工具，它能使用正規表達式搜尋文本，并把比對的行列印出來

可用于資料查找定位

先列舉出測試工作常用的grep指令和意義：

grep pattern file
grep -i pattern file 忽略大小寫
grep -v pattern file 不顯示比對行
grep -o pattern file 隻把每個比對的内容獨立的行顯示
grep -E pattern file 使用拓展正規表達式,相當于egrep
#注意：grep 'a[0-9]\{10\}' 等同于 grep -E 'a[0-9]{10}' 
grep  -B, --before-context=NUM  列印以文本起始的NUM 行
grep  -A, --after-context=NUM   列印以文本結尾的NUM 行
grep  -C, --context=NUM         列印輸出文本NUM 行
grep pattern -r dir/ 遞歸搜尋
grep -m1 比對比對中的第一個
grep -n 順便輸出行号
grep -P 可以讓grep使用perl的正規表達式文法

BRE基本正規表達式：

^開頭 $結尾
[a-z] [0-9]區間，如果開頭帶有^表示不能比對區間内的元素
*0個或多個
.表示任意字元

ERE擴充正規表達式

ERE擴充表達式是基本正規表達式(BRE)基礎上的擴充

？非貪婪比對
+ 一個或多個
() 分組
{} 範圍限制
| 比對多個表達式的任意一個

話不多說，先舉例

假設有如下檔案，名稱為nginx.log

40.77.192.252 - - [05/Dec/2018:00:08:51 +0000] "GET /uploads/photo/2017/ed39a6f0-8111-48bc-88e0-604939b1dca1.png!large HTTP/1.1" 200 10065 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b" 0.000 0.000 .
220.181.108.187 - - [05/Dec/2018:00:09:04 +0000] "GET /topics/15275?locale=en HTTP/1.1" 200 28527 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.090 0.090 .
116.25.42.125 - - [05/Dec/2018:00:09:17 +0000] "GET /cable HTTP/1.1" 101 1473 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3622.0 Safari/537.36" 108.100 108.100 .
123.127.112.18 - - [05/Dec/2018:00:09:18 +0000] "GET /cable HTTP/1.1" 101 1017 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" 70.577 70.577 .
139.180.132.174 - - [05/Dec/2018:00:09:20 +0000] "GET /bbs.zip HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.011 0.011 .
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
141.8.142.131 - - [05/Dec/2018:00:09:12 +0000] "GET /topics/14442 HTTP/1.1" 200 21980 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 0.195 0.195 .
220.181.108.181 - - [05/Dec/2018:00:09:13 +0000] "GET /syyair/following?locale=zh-CN HTTP/1.1" 200 13355 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.035 0.035 .
139.180.132.174 - - [05/Dec/2018:00:09:13 +0000] "GET /admin.rar HTTP/1.1" 401 50 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.006 0.006 .
40.77.167.1 - - [05/Dec/2018:00:09:17 +0000] "GET /topics/2547/replies/17243/reply_suggest HTTP/1.1" 404 3530 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" 0.014 0.014 .
139.180.132.174 - - [05/Dec/2018:00:09:17 +0000] "GET /admin.tar.gz HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.052 0.052 .

找出500錯誤時候的上下文

chenshifengdeMacBook-Pro:~ chenshifeng$ grep -A 2 ' 500 ' nginx.log    #顯示500錯誤及其後2行
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
141.8.142.131 - - [05/Dec/2018:00:09:12 +0000] "GET /topics/14442 HTTP/1.1" 200 21980 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 0.195 0.195 .
220.181.108.181 - - [05/Dec/2018:00:09:13 +0000] "GET /syyair/following?locale=zh-CN HTTP/1.1" 200 13355 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.035 0.035 .
chenshifengdeMacBook-Pro:~ chenshifeng$ grep -B 2 ' 500 ' nginx.log    #顯示500錯誤及其前2行
123.127.112.18 - - [05/Dec/2018:00:09:18 +0000] "GET /cable HTTP/1.1" 101 1017 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" 70.577 70.577 .
139.180.132.174 - - [05/Dec/2018:00:09:20 +0000] "GET /bbs.zip HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.011 0.011 .
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
chenshifengdeMacBook-Pro:~ chenshifeng$ grep -C 2 ' 500 ' nginx.log    #顯示500錯誤及其前後2行
123.127.112.18 - - [05/Dec/2018:00:09:18 +0000] "GET /cable HTTP/1.1" 101 1017 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" 70.577 70.577 .
139.180.132.174 - - [05/Dec/2018:00:09:20 +0000] "GET /bbs.zip HTTP/1.1" 404 1264 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.011 0.011 .
139.180.132.174 - - [05/Dec/2018:00:09:12 +0000] "GET /__zep__/js.zip HTTP/1.1" 500 2183 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" 0.018 0.018 .
141.8.142.131 - - [05/Dec/2018:00:09:12 +0000] "GET /topics/14442 HTTP/1.1" 200 21980 "-" "Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots)" 0.195 0.195 .
220.181.108.181 - - [05/Dec/2018:00:09:13 +0000] "GET /syyair/following?locale=zh-CN HTTP/1.1" 200 13355 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" 0.035 0.035 .

1、先通路部落格首頁，利用 grep href 過濾出所有包含 url 的内容。指令：

chenshifengdeMacBook-Pro:~ chenshifeng$ curl -s https://www.cnblogs.com/feng0815/ | grep -o "https:[^\"]*"
https://www.cnblogs.com/feng0815/rss
https://www.cnblogs.com/feng0815/rsd.xml
https://www.cnblogs.com/feng0815/wlwmanifest.xml
https://common.cnblogs.com/scripts/jquery-2.2.0.min.js
https://mathjax.cnblogs.com/2_7_5/MathJax.js?config=TeX-AMS-MML_HTMLorMML&amp;v=20200504
https://www.cnblogs.com/
https://news.cnblogs.com/
https://q.cnblogs.com/
https://brands.cnblogs.com/
https://ing.cnblogs.com/
https://edu.cnblogs.com/
https://zzk.cnblogs.com/s
https://passport.cnblogs.com/GetBlogApplyStatus.aspx
https://msg.cnblogs.com/
https://home.cnblogs.com/
https://passport.cnblogs.com/GetBlogApplyStatus.aspx
https://home.cnblogs.com/
https://account.cnblogs.com/settings/account
https://account.cnblogs.com/signup/
......

chenshifengdeMacBook-Pro:~ chenshifeng$ curl -s https://www.cnblogs.com/feng0815/ | grep -o "https:[^\"]*"| while read line;do curl -s -I $line | grep ' 200 ' && echo $line || echo ERR $line;done
ERR https://www.cnblogs.com/feng0815/rss
HTTP/1.1 200 OK
https://www.cnblogs.com/feng0815/rsd.xml
HTTP/1.1 200 OK
https://www.cnblogs.com/feng0815/wlwmanifest.xml
HTTP/1.1 200 OK
https://common.cnblogs.com/scripts/jquery-2.2.0.min.js
HTTP/1.1 200 OK
https://mathjax.cnblogs.com/2_7_5/MathJax.js?config=TeX-AMS-MML_HTMLorMML&amp;v=20200504
HTTP/1.1 200 OK
https://www.cnblogs.com/
HTTP/1.1 200 OK
https://news.cnblogs.com/
HTTP/1.1 200 OK
https://q.cnblogs.com/
HTTP/1.1 200 OK
https://brands.cnblogs.com/
HTTP/1.1 200 OK
https://ing.cnblogs.com/
HTTP/1.1 200 OK
https://edu.cnblogs.com/
HTTP/1.1 200 OK
https://zzk.cnblogs.com/s
ERR https://passport.cnblogs.com/GetBlogApplyStatus.aspx
ERR https://msg.cnblogs.com/
HTTP/1.1 200 OK
https://home.cnblogs.com/
ERR https://passport.cnblogs.com/GetBlogApplyStatus.aspx
......

測試必會之 Linux 三劍客之 grep

繼續閱讀

Java String.format方法的簡單使用

neo4j之cypher使用文檔

GitHub連夜封殺！這份阿裡 10W 字内部 Java 字面試手冊到底有多強？

spark/scala關于【資源檔案】加載方法概述外部檔案加載方案測試資源檔案打包入jar包中小結

mybatis_入門程式Mybatis入門

samba伺服器的功能

AOP程式設計_Android優雅權限架構(1)概念基礎，2021金三銀四前言正文大綱正文

Effective Java 8:通用程式設計

【Linux】UDP廣播封包接收速率問題

OOM三種類型

工廠模式-三種類型

【遞歸】高效率求2的n次幂

win10本地scala和spark安裝安裝scala安裝spark

Linux裝置模型（中）之上層容器

scala (3) Function 和 Method

PowerPC平台 Linux移植三