故障現象:
客戶反映園區倉庫通路NC系統速度很慢,從園區ping NC伺服器位址時有嚴重的丢包現象。如下圖所示:
<a href="http://s3.51cto.com/wyfs02/M00/8A/8C/wKiom1gz1DKi117aAAKQFNfloAw338.png-wh_500x0-wm_3-wmp_4-s_1579186669.png" target="_blank"></a>
<a href="http://s1.51cto.com/wyfs02/M02/8A/8C/wKiom1gz1DOx2rwHAAFuqD6gfac287.png-wh_500x0-wm_3-wmp_4-s_2366122665.png" target="_blank"></a>
處理過程:
登入使用者兩個園區的核心發現,兩個園區的portchannel連接配接正常,但是其中一個園區portchannel成員端口下存在嚴重的CRC校驗錯誤,如下圖所示:
<a href="http://s1.51cto.com/wyfs02/M02/8A/89/wKioL1gz08HDzqxmAALCiT0OPh0626.png-wh_500x0-wm_3-wmp_4-s_3053852100.png" target="_blank"></a>
此時,關閉端口ten1/5/1,資料轉發從另一個端口走,關閉後再次PING NC伺服器,資料包正常
<a href="http://s3.51cto.com/wyfs02/M02/8A/89/wKioL1gz1KDgfrB5AAFB4YDNDbs447.png-wh_500x0-wm_3-wmp_4-s_1719984639.png" target="_blank"></a>
<a href="http://s3.51cto.com/wyfs02/M00/8A/8C/wKiom1gz1KCTa0kNAAFlB_3Ct-w146.png-wh_500x0-wm_3-wmp_4-s_3799085379.png" target="_blank"></a>
為進一步确認該故障,再次對ten1/5/1進行開啟操作,但打開後發現,問題立即出現
SW#ping 192.168.x.x repeat 1000 source 192.168.x.x
Type escape sequence to abort.
Sending 1000, 100-byte ICMP Echos to 192.168.x.x, timeout is 2 seconds:
Packet sent with a source address of 192.168.x.x
!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!.!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!.!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!.!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!
!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!.!!!!!!!!!!!!!!!!!..!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!
!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!
Success rate is 97 percent (978/1000), round-trip min/avg/max = 1/2/20 ms
SW#sh int ten1/5/1
TenGigabitEthernet1/5/1 is up, line protocol is up (connected)
Hardware is Ten Gigabit Ethernet Port, address is 78da.6e07.7b7a (bia 78da.6e07.7b7a)
Description: connection to laishanchangqu
MTU 1500 bytes, BW 10000000 Kbit/sec, DLY 10 usec,
reliability 245/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
Full-duplex, 10Gb/s, link type is auto, media type is 10GBase-ER
input flow-control is on, output flow-control is on
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:03, output never, output hang never
Last clearing of "show interface" counters never
Input queue: 0/2000/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 3209000 bits/sec, 358 packets/sec
5 minute output rate 926000 bits/sec, 685 packets/sec
4935914010 packets input, 5278619447870 bytes, 0 no buffer
Received 87068 broadcasts (87050 multicasts)
0 runts, 0 giants, 0 throttles
277020288 input errors, 277020288 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
9064745454 packets output, 3287137727436 bytes, 0 underruns
0 output errors, 0 collisions, 4 interface resets
0 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
SW#conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW(config)#int ten1/5/1
SW(config-if)#shut
SW(config-if)#do sh run int ten1/5/1
Building configuration...
Current configuration : 171 bytes
!
interface TenGigabitEthernet1/5/1
switchport access vlan 133
switchport mode access
shutdown
channel-group 10 mode on
end
SW(config-if)#exit
SW(config)#exit
SW#wr
Compressed configuration from 21267 bytes to 7568 bytes[OK]
Sending 1000, 100-byte ICMP Echos to 192.168.x.x timeout is 2 seconds:
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 1/2/20 ms
經最終确認,NC通路問題主要是由于該端口的CRC校驗錯誤引起。
解決建議:
1、檢查兩廠區之間的鍊路
2、檢查子產品
3、檢查尾纖
*********************
最終:
第二天測試兩個園區之間的鍊路,發現一根光纖的衰減在-21左右,而該接口上的萬兆子產品可接受的衰減最小值是-15.8,後來營運商對線路進行調優檢測,最終恢複正常,測試,網絡通路正常
本文轉自 彎月樓主 51CTO部落格,原文連結:http://blog.51cto.com/05wylz/1875378,如需轉載請自行聯系原作者