APT29 ATT&CK知識庫評測分析指南ATT&CK真實情況分析報告

ATT&CK真實情況分析報告

本報告結合真實資料對ATT&CK的技術矩陣情況和APT29評測進行資料統計分析

In [1]:

from attackcti import attack_client
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

pd.set_option('max_colwidth',3000)
pd.set_option('display.max_rows', None)
pd.set_option('precision',2) 

lift = attack_client()
all_techniques= lift.get_techniques(stix_format=False)
      

ATT&CK矩陣數量

ATT&CK一共有535個技術矩陣，每個技術矩陣都是一個攻防檢測點。

In [2]:

techniques_normalized = pd.json_normalize(all_techniques)
techniques = techniques_normalized.reindex( \
    ['matrix','platform','tactic','technique','technique_id','data_sources'], axis=1)
techniques.shape[0]
      

Out[2]:

535      

ATT&CK的技術矩陣涉及平台分類

ATT&CK現按平台分為windows、linux、macos、office365、saas共5類技術矩陣，其中windows平台涉及222個技術矩陣。

附:柱狀統計圖

In [3]:

platform = {'Windows':'Windows',
            'Linux':'Linux',
            'macOS':'macOS',
            'office365':'Office 365',
            'SaaS':'SaaS'}

counts_list = []
for (data_key,data_str) in platform.items():
    counts_df =  techniques[ (techniques['platform'].apply(str).str.contains(data_str)) ]
    counts_list.append( ( data_key, counts_df.shape[0] ) )

platforms_df = pd.DataFrame( counts_list ).sort_values(1,ascending=False)
platforms_df = platforms_df.rename(columns={0:'platforms',1:'techniques'})

ax = platforms_df.plot( kind='bar',figsize=(10,8), fontsize=20 ,x=0, y=1, rot=360 )
for p in ax.patches:
    ax.annotate(str(p.get_height()), (p.get_x() + 0.1 , p.get_height() + 2 ) ,size= 20 )
plt.show()
      

ATT&CK檢測建議資料源

ATT&CK的每個技術矩陣的檢測，都建議了對應的資料源，其中windows平台建議了56種資料源。安全人員可以參考建議的資料源，檢測ATT&CK技術矩陣歸類的攻擊活動。

In [4]:

win_data = techniques[ 
    (techniques['platform'].apply(str).str.contains('Windows')) 
]
win_data = pd.DataFrame(win_data[['technique','data_sources']])
win_data = pd.DataFrame(win_data.explode('data_sources'))
source_data = win_data.groupby(by='data_sources') \
    .count() \
    .sort_values(by='technique',ascending=True)
source_data.reset_index(inplace=True) 
source_data.shape[0]
      

Out[4]:

56      

56個資料源可以對應檢測出不同的技術矩陣。

1. Process monitoring （程序監控）
2. Process command-line parameters （程序指令參數）
3. File monitoring （檔案讀寫監控）
4. API monitoring （API調用監控）
5. Process use of network （程序網絡關聯）
6. Windows Registry （Windows系統資料庫）
7. Packet capture （本機抓包）
8. Netflow/Enclave netflow （網絡流量）
9. Windows event logs （windows事件日志）
10. Authentication logs （身份認證日志）
11. Network protocol analysis （網絡協定分析）
12. DLL monitoring （DLL加載監控）
13. Binary file metadata （二進制檔案中繼資料）
14. Loaded DLLs （已加載dll檔案）
15. SSL/TLS inspection （SSL/TLS檢查）
16. Azure activity logs （Azure活動日志）
17. PowerShell logs （Powershell日志）
18. Network intrusion detection system （網絡入侵檢測系統）
19. Malware reverse engineering （病毒逆向工程）
20. AWS CloudTrail logs （AWS雲日志）
21. Anti-virus （防毒軟體）
22. Network device logs （網絡裝置日志）
23. Application logs （程式日志）
24. Kernel drivers （驅動檔案）
25. Stackdriver logs （Stackdriver日志）
26. System calls （系統調用）
27. Data loss prevention （資料洩漏防護）
28. Web proxy （網頁代理）
29. Email gateway （郵件網關）
30. Office 365 account logs （Office365賬戶日志）
31. Host network interface （主機網絡接口）
32. User interface （使用者接口）
33. Web logs （網頁日志）
34. Mail server （郵件伺服器）
35. Services （服務）
36. Windows Error Reporting （Windows錯誤報告）
37. Web application firewall logs （網頁防火牆日志）
38. BIOS （主機闆系統）
39. Third-party application logs （第三方程式日志）
40. MBR （磁盤主引導記錄）
41. DNS records （DNS記錄）
42. Detonation chamber （引爆作業）
43. Office 365 trace logs （Office365跟蹤日志）
44. Sensor health and status （傳感器健康狀态）
45. Component firmware （固件元件）
46. VBR （卷引導記錄）
47. Access tokens （通路令牌）
48. Environment variable （環境變量）
49. Asset management （資産管理平台）
50. EFI （可擴充固件接口）
51. Named Pipes （命名管道）
52. Disk forensics （磁盤驗證）
53. WMI Objects （WMI對象）
54. Browser extensions （浏覽器擴充）
55. Digital certificate logs （數字證書日志）
56. OAuth audit logs （OAuth審計日志）

附：56個檢測資料源對應的技術矩陣數橫向柱狀統計

可以看到排名靠前的程序監控、程序指令參數、檔案讀寫監控、API調用監控、程序網絡關聯等這些EDR類産品的重點關注資料。

In [5]:

tlist = source_data['data_sources'].tolist()
ax = source_data.plot(kind='barh',figsize=(110,70),x=0, y=1, fontsize=65)
for i in ax.patches:
    ax.text(i.get_width(), i.get_y(), str(i.get_width()), fontsize=70)
my_x_ticks = np.arange(0, 160, 40)
plt.xticks(my_x_ticks)
plt.show()
      

ATT&CK的APT29攻擊模拟評估

此次評估一共有21家安全廠商參與，評估資料公開透明，非常适合安全人員分析研究安全廠商的真實能力。

資料來源： https://attackevals.mitre.org

In [6]:

import json,glob,os

files =[]
for infile in sorted(glob.glob(os.path.join('data', '*json'))):
   files.append(infile)
print(len(files))
files
      

21
      

Out[6]:

['data/Bitdefender.1.APT29.1_Results.json',
 'data/CrowdStrike.1.APT29.1_Results.json',
 'data/Cybereason.1.APT29.1_Results.json',
 'data/Cycraft.1.APT29.1_Results.json',
 'data/Cylance.1.APT29.1_Results.json',
 'data/Elastic.1.APT29.1_Results.json',
 'data/F-Secure.1.APT29.1_Results.json',
 'data/FireEye.1.APT29.1_Results.json',
 'data/GoSecure.1.APT29.1_Results.json',
 'data/HanSight.1.APT29.1_Results.json',
 'data/Kaspersky.1.APT29.1_Results.json',
 'data/Malwarebytes.1.APT29.1_Results.json',
 'data/McAfee.1.APT29.1_Results.json',
 'data/Microsoft.1.APT29.1_Results.json',
 'data/PaloAltoNetworks.1.APT29.1_Results.json',
 'data/ReaQta.1.APT29.1_Results.json',
 'data/Secureworks.1.APT29.1_Results.json',
 'data/SentinelOne.1.APT29.1_Results.json',
 'data/Symantec.1.APT29.1_Results.json',
 'data/TrendMicro.1.APT29.1_Results.json',
 'data/VMware.1.APT29.1_Results.json']      

In [7]:

from natsort import index_natsorted, order_by_index
import copy

all_data = {}

for f_path in files:
    vendor = f_path.split(os.sep, 2)[-1]
    vendor = vendor.split('.', 1)[0]
    with open(f_path, 'r', encoding='utf-8') as infile:
        data=infile.read()
        obj = json.loads(data)['Techniques']
        df = pd.json_normalize(obj,'Steps', ['TechniqueId','TechniqueName', 'Tactics'])
        all_data.update({ vendor: df }) 
      

APT29評估方法

此次評估一共有140個攻擊動作步驟，每個步驟對應不同的戰術Tactics和技術Technique。

• 戰術Tactics （攻擊動作的意圖分類）
• 技術Technique （攻擊動作的技術分類）
• 标準Criteria （攻擊動作的詳細過程）
• 程式Procedure （攻擊動作的技術細節）

In [8]:

test_data = copy.deepcopy(all_data)    
test_data = test_data.values()
test_data = list(test_data)[0]
eval_step = test_data.reindex(index=order_by_index(test_data.index, 
                         index_natsorted(test_data['SubStep'])
                        ))
eval_step.reset_index(drop=True, inplace=True)   
eval_step['TacticsName']=eval_step['Tactics'].apply(lambda x: x[0]['TacticName']) 
eval_step = eval_step.reindex(['SubStep','TacticsName','TechniqueName','Criteria','Procedure'], axis=1) 
eval_step
      

Out[8]:

SubStep	TacticsName	TechniqueName	Criteria	Procedure
1.A.1	Execution	User Execution	The rcs.3aka3.doc process spawning from explorer.exe	User Pam executed payload rcs.3aka3.doc
1	1.A.2	Defense Evasion	Masquerading	Evidence of the right-to-left override character (U+202E) in the rcs.3aka.doc process ​OR the original filename (cod.3aka.scr)	Used unicode right-to-left override (RTLO) character to obfuscate file name rcs.3aka3.doc (originally cod.3aka.scr)
2	1.A.3	Command and Control	Uncommonly Used Port	Established network channel over port 1234	Established C2 channel (192.168.0.5) via rcs.3aka3.doc payload over TCP port 1234
3	1.A.4	Command and Control	Standard Cryptographic Protocol	Evidence that the network data sent over the C2 channel is encrypted	Used RC4 stream cipher to encrypt C2 (192.168.0.5) traffic
4	1.B.1	Execution	Command-Line Interface	cmd.exe spawning from the rcs.3aka3.doc​ process	Spawned interactive cmd.exe
5	1.B.2	Execution	PowerShell	powershell.exe spawning from cmd.exe	Spawned interactive powershell.exe
6	2.A.1	Discovery	File and Directory Discovery	powershell.exe executing (Get-)ChildItem	Searched filesystem for document and media files using PowerShell
7	2.A.2	Collection	Automated Collection	powershell.exe executing (Get-)ChildItem	Scripted search of filesystem for document and media files using PowerShell
8	2.A.3	Collection	Data from Local System	powershell.exe reading files in C:\Users\Pam\	Recursively collected files found in C:\Users\Pam\ using PowerShell
9	2.A.4	Exfiltration	Data Compressed	powershell.exe executing Compress-Archive	Compressed and stored files into ZIP (Draft.zip) using PowerShell
10	2.A.5	Collection	Data Staged	powershell.exe creating the file draft.zip	Staged files for exfiltration into ZIP (Draft.zip) using PowerShell
11	2.B.1	Exfiltration	Exfiltration Over Command and Control Channel	The rcs.3aka3.doc process reading the file draft.zip while connected to the C2 channel	Read and downloaded ZIP (Draft.zip) over C2 channel (192.168.0.5 over TCP port 1234)
12	3.A.1	Command and Control	Remote File Copy	The rcs.3aka3.doc process creating the file monkey.png	Dropped stage 2 payload (monkey.png) to disk
13	3.A.2	Defense Evasion	Obfuscated Files or Information	Evidence that a PowerShell payload was within monkey.png	Embedded PowerShell payload in monkey.png using steganography
14	3.B.1	Defense Evasion	Component Object Model Hijacking	Addition of the DelegateExecute ​subkey in ​HKCU\Software\Classes\Folder\shell\open\​​command​​	Modified the Registry to enable COM hijacking of sdclt.exe using PowerShell
15	3.B.2	Privilege Escalation	Bypass User Account Control	High integrity powershell.exe spawning from control.exe​​ (spawned from sdclt.exe)	Executed elevated PowerShell payload
16	3.B.3	Command and Control	Commonly Used Port	Established network channel over port 443	Established C2 channel (192.168.0.5) via PowerShell payload over TCP port 443
17	3.B.4	Command and Control	Standard Application Layer Protocol	Evidence that the network data sent over the C2 channel is HTTPS	Used HTTPS to transport C2 (192.168.0.5) traffic
18	3.B.5	Command and Control	Standard Cryptographic Protocol	Evidence that the network data sent over the C2 channel is encrypted	Used HTTPS to encrypt C2 (192.168.0.5) traffic
19	3.C.1	Defense Evasion	Modify Registry	Deletion of of the HKCU\Software\Classes\Folder\shell\Open\command subkey	Modified the Registry to remove artifacts of COM hijacking
20	4.A.1	Command and Control	Remote File Copy	powershell.exe creating the file SysinternalsSuite.zip	Dropped additional tools (SysinternalsSuite.zip) to disk over C2 channel (192.168.0.5)
21	4.A.2	Execution	PowerShell	powershell.exe spawning from powershell.exe	Spawned interactive powershell.exe
22	4.A.3	Defense Evasion	Deobfuscate/Decode Files or Information	powershell.exe executing Expand-Archive	Decompressed ZIP (SysinternalsSuite.zip) file using PowerShell
23	4.B.1	Discovery	Process Discovery	powershell.exe executing Get-Process	Enumerated current running processes using PowerShell
24	4.B.2	Defense Evasion	File Deletion	sdelete64.exe deleting the file rcs.3aka3.doc	Deleted rcs.3aka3.doc on disk using SDelete
25	4.B.3	Defense Evasion	File Deletion	sdelete64.exe deleting the file draft.zip	Deleted Draft.zip on disk using SDelete
26	4.B.4	Defense Evasion	File Deletion	sdelete64.exe deleting the file SysinternalsSuite.zip	Deleted SysinternalsSuite.zip on disk using SDelete
27	4.C.1	Discovery	File and Directory Discovery	powershell.exe executing $env:TEMP	Enumerated user's temporary directory path using PowerShell
28	4.C.2	Discovery	System Owner/User Discovery	powershell.exe executing $env:USERNAME	Enumerated the current username using PowerShell
29	4.C.3	Discovery	System Information Discovery	powershell.exe executing $env:COMPUTERNAME	Enumerated the computer hostname using PowerShell
30	4.C.4	Discovery	System Network Configuration Discovery	powershell.exe executing $env:USERDOMAIN	Enumerated the current domain name using PowerShell
31	4.C.5	Discovery	Process Discovery	powershell.exe executing $PID	Enumerated the current process ID using PowerShell
32	4.C.6	Discovery	System Information Discovery	powershell.exe executing​ Gwmi Win32_OperatingSystem	Enumerated the OS version using PowerShell
33	4.C.7	Discovery	Security Software Discovery	powershell.exe executing​ Get-WmiObject ...​ -Class AntiVirusProduct	Enumerated anti-virus software using PowerShell
34	4.C.8	Discovery	Security Software Discovery	powershell.exe executing Get-WmiObject ...​​ -Class FireWallProduct	Enumerated firewall software using PowerShell
35	4.C.9	Discovery	Permission Groups Discovery	powershell.exe executing the NetUserGetGroups API	Enumerated user's domain group membership via the NetUserGetGroups API
36	4.C.10	Execution	Execution through API	The NetUserGetGroups API function loaded into powershell.exe from Netapi32.dll	Executed API call by reflectively loading Netapi32.dll
37	4.C.11	Discovery	Permission Groups Discovery	powershell.exe executing the NetUserGetLocalGroups API	Enumerated user's local group membership via the NetUserGetLocalGroups API
38	4.C.12	Execution	Execution through API	The NetUserGetLocalGroups API function loaded into powershelle.exe from Netapi32.dll	Executed API call by reflectively loading Netapi32.dll
39	5.A.1	Persistence	New Service	powershell.exe creating the Javamtsup service	Created a new service (javamtsup) that executes a service binary (javamtsup.exe) at system startup
40	5.B.1	Persistence	Registry Run Keys / Startup Folder	powershell.exe creating the file hostui.lnk in the Startup folder	Created a LNK file (hostui.lnk) in the Startup folder that executes on login
41	6.A.1	Credential Access	Credentials in Files	accesschk.exe reading files within %APPDATALOCAL%\Google\chrome\user data\default\	Read the Chrome SQL database file to extract encrypted credentials
42	6.A.2	Credential Access	Credential Dumping	accesschk.exe executing the CryptUnprotectedData API	Executed the CryptUnprotectedData API call to decrypt Chrome passwords
43	6.A.3	Defense Evasion	Masquerading	Evidence that accesschk.exe is not the legitimate Sysinternals tool	Masqueraded a Chrome password dump tool as accesscheck.exe, a legitimate Sysinternals tool
44	6.B.1	Credential Access	Private Keys	powershell.exe creating a certificate file exported from the system	Exported a local certificate to a PFX file using PowerShell
45	6.C.1	Credential Access	Credential Dumping	powershell.exe injecting into lsass.exe OR lsass.exe reading Registry keys under HKLM:\SAM\SAM\Domains\Account\Users\	Dumped password hashes from the Windows Registry by injecting a malicious DLL into Lsass.exe
46	7.A.1	Collection	Screen Capture	powershell.exe executing the CopyFromScreen function from System.Drawing.dll	Captured and saved screenshots using PowerShell
47	7.A.2	Collection	Clipboard Data	powershell.exe executing Get-Clipboard	Captured clipboard contents using PowerShell
48	7.A.3	Collection	Input Capture	powershell.exe executing the GetAsyncKeyState API	Captured user keystrokes using the GetAsyncKeyState API
49	7.B.1	Collection	Data from Local System	powershell.exe reading files in C:\Users\pam\Downloads\	Read data in the user's Downloads directory using PowerShell
50	7.B.2	Exfiltration	Data Compressed	powershell.exe creating the file OfficeSupplies.7z	Compressed data from the user's Downloads directory into a ZIP file (OfficeSupplies.7z) using PowerShell
51	7.B.3	Exfiltration	Data Encrypted	powershell.exe executing Compress-7Zip with the password argument used for encryption	Encrypted data from the user's Downloads directory using PowerShell
52	7.B.4	Exfiltration	Exfiltration Over Alternative Protocol	powershell executing Copy-Item pointing to an attack-controlled WebDav network share (192.168.0.4:80)	Exfiltrated collection (OfficeSupplies.7z) to WebDAV network share using PowerShell
53	8.A.1	Discovery	Remote System Discovery	powershell.exe making LDAP queries over port 389 to the Domain Controller (10.0.0.4)	Enumerated remote systems using LDAP queries
54	8.A.2	Execution	Windows Remote Management	Network connection to Scranton (10.0.1.4) over port 5985	Established WinRM connection to remote host Scranton (10.0.1.4)
55	8.A.3	Discovery	Process Discovery	powershell.exe executing Get-Process	Enumerated processes on remote host Scranton (10.0.1.4) using PowerShell
56	8.B.1	Command and Control	Remote File Copy	The file python.exe created on Scranton (10.0.1.4)	Copied python.exe payload from a WebDAV share (192.168.0.4) to remote host Scranton (10.0.1.4)
57	8.B.2	Defense Evasion	Software Packing	Evidence that the file python.exe is packed	python.exe payload was packed with UPX
58	8.C.1	Defense Evasion	Valid Accounts	Successful logon as user Pam on Scranton (10.0.1.4)	Logged on to remote host Scranton (10.0.1.4) using valid credentials for user Pam
59	8.C.2	Lateral Movement	Windows Admin Shares	SMB session to Scanton (10.0.1.4) over TCP port 445/135 OR evidence of usage of a Windows share	Established SMB session to remote host Scranton's (10.0.1.4) IPC$ share using PsExec
60	8.C.3	Execution	Service Execution	python.exe spawned by PSEXESVC.exe	Executed python.exe using PSExec
61	9.A.1	Command and Control	Remote File Copy	python.exe creating the file rar.exe	Dropped rar.exe to disk on remote host Scranton (10.0.1.4)
62	9.A.2	Command and Control	Remote File Copy	python.exe creating the file sdelete64.exe	Dropped sdelete.exe to disk on remote host Scranton (10.0.1.4)
63	9.B.1	Execution	PowerShell	powershell.exe​ spawning from python.exe	Spawned interactive powershell.exe
64	9.B.2	Discovery	File and Directory Discovery	powershell.exe executing (Get-)ChildItem​	Searched filesystem for document and media files using PowerShell
65	9.B.3	Collection	Automated Collection	powershell.exe executing (Get-)ChildItem​	Scripted search of filesystem for document and media files using PowerShell
66	9.B.4	Collection	Data from Local System	powershell.exe reading files in C:\Users\Pam\	Recursively collected files found in C:\Users\Pam\ using PowerShell
67	9.B.5	Collection	Data Staged	powershell.exe creating the file working.zip	Staged files for exfiltration into ZIP (working.zip in AppData directory) using PowerShell
68	9.B.6	Exfiltration	Data Encrypted	powershell.exe executing rar.exe with the -a parameter for a password to use for encryption	Encrypted staged ZIP (working.zip in AppData directory) into working.zip (on Desktop) using rar.exe
69	9.B.7	Exfiltration	Data Compressed	powershell.exe executing rar.exe	Compressed staged ZIP (working.zip in AppData directory) into working.zip (on Desktop) using rar.exe
70	9.B.8	Exfiltration	Exfiltration Over Command and Control Channel	python.exe reading the file working.zip while connected to the C2 channel	Read and downloaded ZIP (working.zip on Desktop) over C2 channel (192.168.0.5 over TCP port 8443)
71	9.C.1	Defense Evasion	File Deletion	sdelete64.exe deleting the file rar.exe	Deleted rar.exe on disk using SDelete
72	9.C.2	Defense Evasion	File Deletion	sdelete64.exe deleting the file \Desktop\working.zip	Deleted working.zip (from Desktop) on disk using SDelete
73	9.C.3	Defense Evasion	File Deletion	sdelete64.exe deleting the file \AppData\Roaming\working.zip	Deleted working.zip (from AppData directory) on disk using SDelete
74	9.C.4	Defense Evasion	File Deletion	cmd.exe deleting the file sdelete64.exe	Deleted SDelete on disk using cmd.exe del command
75	10.A.1	Execution	Service Execution	javamtsup.exe spawning from services.exe	Executed persistent service (javamtsup) on system startup
76	10.B.1	Persistence	Registry Run Keys / Startup Folder	Evidence that the file hostui.lnk (which executes hostui.bat as a byproduct) was executed from the Startup Folder	Executed LNK payload (hostui.lnk) in Startup Folder on user login
77	10.B.2	Execution	Execution through API	hostui.exe executing the\nCreateProcessWithToken API	Executed PowerShell payload via the CreateProcessWithToken API
78	10.B.3	Defense Evasion	Access Token Manipulation	hostui.exe manipulating the token of powershell.exe via the CreateProcessWithToken API OR \npowershell.exe executing with the stolen token of explorer.exe	Manipulated the token of the PowerShell payload via the CreateProcessWithToken API
79	11.A.1	Execution	User Execution	powershell.exe spawning from explorer.exe	User Oscar executed payload 37486-the-shocking-truth-about-election-rigging-in-america.rtf.lnk
80	11.A.2	Defense Evasion	NTFS File Attributes	powershell.exe executing the schemas ADS via Get-Content and IEX	Executed an alternate data stream (ADS) using PowerShell
81	11.A.3	Discovery	Virtualization/Sandbox Evasion	powershell.exe executing a Get-WmiObject\nquery for Win32_BIOS	Checked that the BIOS version and serial number are not associated with VirtualBox or VMware using PowerShell
82	11.A.4	Discovery	System Information Discovery	powershell.exe executing a Get-WmiObject gwmi queries for Win32_BIOS and Win32_ComputerSystem	Enumerated computer manufacturer, model, and version information using PowerShell
83	11.A.5	Discovery	Peripheral Device Discovery	powershell.exe executing a Get-WmiObject query for Win32_PnPEntity	Enumerated devices/adapters to check for presence of VirtualBox driver(s) using PowerShell
84	11.A.6	Discovery	System Owner/User Discovery	powershell.exe executing a Get-WmiObject query for Win32_ComputerSystem	Checked that the username is not related to admin or a generic value (ex: user) using PowerShell
85	11.A.7	Discovery	System Network Configuration Discovery	powershell.exe executing a Get-WmiObject query for Win32_ComputerSystem	Checked that the computer is joined to a domain using PowerShell
86	11.A.8	Discovery	Process Discovery	powershell.exe executing a Get-WmiObject query for Win32_Process	Checked that processes such as procexp.exe, taskmgr.exe, or wireshark.exe are not running using PowerShell
87	11.A.9	Discovery	File and Directory Discovery	powershell.exe executing (Get-Item -Path ".\" -Verbose).FullName	Checked that the payload is not inside a folder path that contains "sample" or is the length of a hash value using PowerShell
88	11.A.10	Defense Evasion	Deobfuscate/Decode Files or Information	certutil.exe decoding kxwn.lock	Decoded an embedded DLL payload to disk using certutil.exe
89	11.A.11	Persistence	Registry Run Keys / Startup Folder	Addition of the Webcache subkey in HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Run	Established Registry Run key persistence using PowerShell
90	11.A.12	Execution	PowerShell	powershell.exe spawning from from the schemas ADS (powershell.exe)	Executed PowerShell stager payload
91	11.A.13	Command and Control	Commonly Used Port	Established network channel over port 443	Established C2 channel (192.168.0.4) via PowerShell payload over port 443
92	11.A.14	Command and Control	Standard Application Layer Protocol	Established network channel over the HTTPS protocol	Used HTTPS to transport C2 (192.168.0.4) traffic
93	11.A.15	Command and Control	Standard Cryptographic Protocol	Evidence that the network data sent over the C2 channel is encrypted	Used HTTPS to encrypt C2 (192.168.0.4) traffic
94	12.A.1	Discovery	File and Directory Discovery	powershell.exe executing (gci ((gci env:windir).Value + '\system32')	Enumerated the System32 directory using PowerShell
95	12.A.2	Defense Evasion	Timestomp	powershell.exe modifying the creation, last access, and last write times of kxwn.lock	Modified the time attributes of the kxwn.lock persistence payload using PowerShell
96	12.B.1	Discovery	Security Software Discovery	powershell.exe executing a Get-WmiObject query for AntiVirusProduct	Enumerated registered AV products using PowerShell
97	12.C.1	Discovery	Query Registry	powershell.exe executing a Registry query for HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall	Enumerated installed software via the Registry (Wow6432 Uninstall key) using PowerShell
98	12.C.2	Discovery	Query Registry	powershell.exe executing a Registry query for HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall	Enumerated installed software via the Registry (Uninstall key) using PowerShell
99	13.A.1	Discovery	System Information Discovery	powershell.exe executing the GetComputerNameEx API	Enumerated the computer name using the GetComputerNameEx API
100	13.B.1	Discovery	System Network Configuration Discovery	powershell.exe executing the NetWkstaGetInfo API	Enumerated the domain name using the NetWkstaGetInfo API
101	13.C.1	Discovery	System Owner/User Discovery	powershell.exe executing the GetUserNameEx API	Enumerated the current username using the GetUserNameEx API
102	13.D.1	Discovery	Process Discovery	powershell.exe executing the CreateToolhelp32Snapshot API	Enumerated running processes using the CreateToolhelp32Snapshot API
103	14.A.1	Defense Evasion	Component Object Model Hijacking	Addition of the DelegateExecute subkey in HKCU\Software\Classes\Folder\shell\open\command	Modified the Registry to enable COM hijacking of sdclt.exe using PowerShell
104	14.A.2	Privilege Escalation	Bypass User Account Control	High integrity powrshell.exe spawning from control.exe​​ (spawned from sdclt.exe)	Executed elevated PowerShell payload
105	14.A.3	Defense Evasion	Modify Registry	Deletion of the HKCU\Software\Classes\Folder\shell\Open\command subkey	Modified the Registry to remove artifacts of COM hijacking using PowerShell
106	14.B.1	Execution	Windows Management Instrumentation	WMI Process (WmiPrvSE.exe) executing powershell.exe	Created and executed a WMI class using PowerShell
107	14.B.2	Discovery	Process Discovery	powershell.exe executing Get-Process	Enumerated and tracked PowerShell processes using PowerShell
108	14.B.3	Command and Control	Remote File Copy	powershell.exe downloading and/or the file write of m.exe	Downloaded and dropped Mimikatz (m.exe) to disk
109	14.B.4	Credential Access	Credential Dumping	m.exe injecting into lsass.exe to dump credentials	Dumped plaintext credentials using Mimikatz (m.exe)
110	14.B.5	Defense Evasion	Obfuscated Files or Information	powershell.exe executing Set-WmiInstance	Encoded and wrote Mimikatz output to a WMI class property using PowerShell
111	14.B.6	Defense Evasion	Deobfuscate/Decode Files or Information	powershell.exe executing Get-WmiInstance	Read and decoded Mimikatz output from a WMI class property using PowerShell
112	15.A.1	Discovery	System Owner/User Discovery	powershell.exe executing $env:UserName	Enumerated logged on users using PowerShell
113	15.A.2	Persistence	Windows Management Instrumentation Event Subscription	powershell.exe creating the WindowsParentalControlMigration WMI filter, consumer, and binding created in root/subscription	Established WMI event subscription persistence using PowerShell
114	16.A.1	Discovery	Remote System Discovery	powershell.exe making LDAP queries over port 389 via functions from System.DirectoryServices.dll	Enumerated the domain controller host NewYork (10.0.0.4) using LDAP queries
115	16.B.1	Discovery	System Owner/User Discovery	powershell.exe executing the ConvertSidToStringSid API	Enumerated the domain SID (from current user SID) using the ConvertSidToStringSid API
116	16.B.2	Execution	Execution through API	powershell.exe executing the ConvertSidToStringSid API function by loading Advapi32.dll	Executed the ConvertSidToStringSid API call by reflectively loading Advapi32.dll
117	16.C.1	Execution	Windows Remote Management	Network connection to NewYork (10.0.0.4) over port 5985	Established a WinRM connection to the domain controller host NewYork (10.0.0.4)
118	16.C.2	Defense Evasion	Valid Accounts	Successful logon as user MScott on NewYork (10.0.0.4)	Logged on to the domain controller host NewYork (10.0.0.4) using valid credentials for user MScott
119	16.D.1	Command and Control	Remote File Copy	File write of m.exe by the WinRM process (wsmprovhost.exe)	Dropped Mimikatz (m.exe) to disk on the domain controller host NewYork (10.0.0.4) over a WinRM connection
120	16.D.2	Credential Access	Credential Dumping	m.exe injecting into lsass.exe to dump credentials	Dumped the KRBTGT hash on the domain controller host NewYork (10.0.0.4) using Mimikatz (m.exe)
121	17.A.1	Collection	Email Collection	outlook.exe spawning from svchost.exe or powershell.exe	Dumped messages from the local Outlook inbox using PowerShell
122	17.B.1	Collection	Data from Local System	powershell.exe reading the file MITRE-ATTACK-EVALS.HTML	Read and collected a local file using PowerShell
123	17.B.2	Collection	Data Staged	powershell.exe creating the file \WindowsParentalControlMigration\MITRE-ATTACK-EVALS.HTML	Staged collected file into directory using PowerShell
124	17.C.1	Exfiltration	Data Compressed	powershell.exe executing the ZipFile.CreateFromDirectory .NET method	Compressed a staging directory using PowerShell
125	17.C.2	Defense Evasion	Obfuscated Files or Information	powershell.exe executing Set-Content	Prepended the GIF file header to a compressed staging file using PowerShell
126	18.A.1	Defense Evasion	Web Service	net.exe with command-line arguments then making a network connection to a public IP over port 443	Mapped a network drive to an online OneDrive account using PowerShell
127	18.A.2	Exfiltration	Exfiltration Over Alternative Protocol	powershell.exe executing Copy-Item pointing to drive mapped to an attack-controlled OneDrive account	Exfiltrated staged collection to an online OneDrive account using PowerShell
128	19.A.1	Defense Evasion	File Deletion	File delete event for C:\Windows\System32\m.exe	Deleted Mimikatz (m.exe) on disk using SDelete
129	19.A.2	Defense Evasion	Process Injection	Injection into PowerShell via Invoke-ReflectivePEInjection	Reflectively injected SDelete binary into PowerShell
130	19.B.1	Defense Evasion	File Deletion	File delete event for C:\Windows\Temp\WindowsParentalControlMigration.tmp	Deleted exfiltrated data on disk using SDelete
131	19.B.2	Defense Evasion	Process Injection	Injection into PowerShell via Invoke-ReflectivePEInjection	Reflectively injected SDelete binary into PowerShell
132	19.C.1	Defense Evasion	File Deletion	File delete event for C:\Windows\Temp\WindowsParentalControlMigration\MITRE-ATTACK-EVALS.HTML	Deleted staged data on disk using SDelete
133	19.C.2	Defense Evasion	Process Injection	Injection into PowerShell via Invoke-ReflectivePEInjection	Reflectively injected SDelete binary into PowerShell
134	20.A.1	Execution	Rundll32	rundll32.exe executing kxwn.lock	Executed Run key persistence payload on user login using RunDll32
135	20.A.2	Persistence	Windows Management Instrumentation Event Subscription	The WMI process (wmiprvse.exe) executing powershell.exe	Executed WMI persistence on user login
136	20.A.3	Execution	PowerShell	SYSTEM-level powershell.exe spawned from the powershell.exe	Executed PowerShell payload from WMI event subscription persistence
137	20.B.1	Lateral Movement	Pass the Ticket	powershell.exe executing Invoke-Mimikatz with command-line arguments to create a golden ticket	Created Kerberos Golden Ticket using Invoke-Mimikatz
138	20.B.2	Execution	Windows Remote Management	Network connection to Scranton (10.0.1.4) over port 5985	Established a WinRM connection to the remote host Scranton (10.0.1.4) using the Golden Ticket as credentials
139	20.B.3	Persistence	Create Account	net.exe adding the user Toby	Added a new user to the remote host Scranton (10.0.1.4) using net.exe

關于如何評測APT29模拟攻擊的檢出資料

APT29攻擊的測試環境工具已經完全開源 https://github.com/mitre-attack/attack-arsenal

廠商的檢出結果公開透明，MITRE ATT&CK官方的評估标準顆粒度較細，檢出結果類型分主要檢測類型和修飾檢測類型。修飾檢測類型相當于附加描叙,可以算是加分項。

由于MITRE ATT&CK官方隻給出統計資料，而各方的評分标準不統一，導緻各路分析師和廠商對應檢測成績的評定過于兩極化，同時如果過多關注檢測類型組合情況反而複雜化了評估，反應不出廠商真實水準，是以我以一線技術人員的了解給出了評估方法。

主要按如下方法評估廠商

• 非None類型都可認為是檢出情況，可統計技術矩陣覆寫度。
• Telemetry和MSSP兩種類型告警進行統計，可統計需要分析師營運跟進的模糊告警資料。
• General、Tactic、Technique三種類型進行統計，可統計明确的惡意告警資料。
• 按主要檢測類型的價值梯度進行打分，以檢出結果的營運價值得出廠商排名。

主要檢測類型

• None （無檢出）
• Telemetry （遙測型檢出，僅作為打點資料，無上下文參考或明确惡意标記的告警）
• MSSP （安全營運型檢出，需要分析師進一步關聯分析判斷才能确認威脅的告警）
• General （通用型檢出，标記為通用的惡意行為告警，無準确的技戰術分類）
• Tactic （戰術型檢出，向分析師提供了攻擊動作潛在意圖資訊的告警）
• Technique （技術型檢出，向分析師提供了攻擊動作的詳細技戰術資訊的告警）

修飾檢測類型

• Alert （告警）
• Correlated （相關）
• Delayed （延遲）
• Host Interrogation （主機響應）
• Residual Artifact （殘留工件）
• Configuration Change （配置更改）
• Innovative （創新檢測）

此次APT29評估一共涉及了ATT&CK共58個技術矩陣，每個技術矩陣涉及多個模拟攻擊動作步驟。

附：統計清單

In [9]:

tech_count = eval_step.reindex(['TechniqueName','SubStep'],axis=1)
tech_count = tech_count.groupby(['TechniqueName']).count(). \
    sort_values(by='SubStep',ascending=False)
tech_count.reset_index(inplace=True)  
tech_count
#tech_count.to_csv('tech_count.csv',encoding='utf_8_sig')
      

Out[9]:

TechniqueName	SubStep
File Deletion	10
1	Remote File Copy	7
2	Process Discovery	6
3	PowerShell	5
4	System Owner/User Discovery	5
5	File and Directory Discovery	5
6	Execution through API	4
7	System Information Discovery	4
8	Credential Dumping	4
9	Data Compressed	4
10	Data from Local System	4
11	Obfuscated Files or Information	3
12	Process Injection	3
13	Registry Run Keys / Startup Folder	3
14	Security Software Discovery	3
15	Windows Remote Management	3
16	Data Staged	3
17	Standard Cryptographic Protocol	3
18	System Network Configuration Discovery	3
19	Deobfuscate/Decode Files or Information	3
20	Service Execution	2
21	Automated Collection	2
22	Remote System Discovery	2
23	Standard Application Layer Protocol	2
24	Query Registry	2
25	User Execution	2
26	Valid Accounts	2
27	Windows Management Instrumentation Event Subscription	2
28	Permission Groups Discovery	2
29	Data Encrypted	2
30	Component Object Model Hijacking	2
31	Bypass User Account Control	2
32	Modify Registry	2
33	Masquerading	2
34	Exfiltration Over Command and Control Channel	2
35	Exfiltration Over Alternative Protocol	2
36	Commonly Used Port	2
37	Windows Management Instrumentation	1
38	Windows Admin Shares	1
39	Web Service	1
40	Virtualization/Sandbox Evasion	1
41	Clipboard Data	1
42	Command-Line Interface	1
43	Uncommonly Used Port	1
44	Timestomp	1
45	Peripheral Device Discovery	1
46	Create Account	1
47	Credentials in Files	1
48	Pass the Ticket	1
49	Software Packing	1
50	Email Collection	1
51	Screen Capture	1
52	Rundll32	1
53	Input Capture	1
54	NTFS File Attributes	1
55	New Service	1
56	Private Keys	1
57	Access Token Manipulation	1

挑一個廠商的File Deletion技術矩陣檢出日志檢視一下檢出類型

檢出類型有Telemetry類型，有MSSP類型，也有Technique類型，以及具有争議的N/A無類型檢出。

In [10]:

from IPython.display import JSON
import warnings
warnings.filterwarnings('ignore')

cy = all_data['Cycraft'][ 
    (all_data['Cycraft']['TechniqueName']=='File Deletion') 
]

print(list(all_data.keys()))
JSON(cy['Detections'].to_json(),expanded=True)
      

['Bitdefender', 'CrowdStrike', 'Cybereason', 'Cycraft', 'Cylance', 'Elastic', 'F-Secure', 'FireEye', 'GoSecure', 'HanSight', 'Kaspersky', 'Malwarebytes', 'McAfee', 'Microsoft', 'PaloAltoNetworks', 'ReaQta', 'Secureworks', 'SentinelOne', 'Symantec', 'TrendMicro', 'VMware']
      

Out[10]:

<IPython.core.display.JSON object>      

APT29評估檢出覆寫度統計

對應140個步驟的檢測結果，在不考慮檢測類型即準确性和誤報的情況下,僅對廠商有檢出結果的技術矩陣覆寫度進行統計排名。

結論： 絕大多數廠商都已經覆寫了100個以上攻擊動作步驟的檢出，說明各廠商産品對ATT&CK技術矩陣分類的攻擊動作相關告警跟進力度都很大。

In [11]:

def check_all_Detection():  
    
    flag_list = []
    flag_data = copy.deepcopy(all_data)
    
    def check_deep_Detection(DetectionNote):
        Detection = 0
        for xl in DetectionNote:
            if xl['DetectionType']=='None':
                Detection = 0
            else:
                Detection = +1
        return Detection

    for i,d in flag_data.items():
        d['Detections'] =  d['Detections'].apply(lambda x: check_deep_Detection(x)) 
        d_count =d['Detections'].sum()       
        flag_list.append( (i,d_count) )
    return flag_list

flag_list =check_all_Detection()
flag_df = pd.DataFrame( flag_list ).sort_values(1,ascending=True)
ax = flag_df.plot(kind='barh',figsize=(110,70),x=0, y=1, fontsize=65)
for i in ax.patches:
    #ax.text(i.get_width(), i.get_y() + 0.2, '{:.0%}'.format(i.get_width()/140), fontsize=70)
    ax.text(i.get_width(), i.get_y() + 0.2, i.get_width() , fontsize=70)
my_x_ticks = np.arange(0, 160, 40)
plt.xticks(my_x_ticks)
plt.show()
      

APT29評估遙測和安全營運類型檢出資料統計

廠商的遙測營運能力統計，隻統計每個攻擊動作步驟中廠商檢出的Telemetry和MSSP類型告警資料。

結論：在140個攻擊動作步驟檢出結果裡，廠商的檢出資料絕大多數都是遙測營運類資料，這類檢出資料都需要分析師進一步關聯分析才能确認威脅,可以看出廠商的産品路線都是在走分析師參與的重營運路線。

In [12]:

def check_custom_Detection( check_list ):    
    flag_list = []
    flag_data = copy.deepcopy(all_data)
    
    def check_deep1_Detection(DetectionNote):
        Detection = 0
        for xl in DetectionNote:
            for cl in check_list:
                if xl['DetectionType']== cl:
                    Detection = +1
        return Detection

    for i,d in flag_data.items():
        d['Detections'] =  d['Detections'].apply(lambda x: check_deep1_Detection(x)) 
        d_count =d['Detections'].sum()   
        flag_list.append( (i,d_count) )
        
    tel_df = pd.DataFrame( flag_list ).sort_values(1,ascending=True).round(1)
    ax = tel_df.plot(kind='barh',figsize=(110,70),x=0, y=1, fontsize=65)
    for i in ax.patches:
        ax.text(i.get_width(), i.get_y() + 0.2, i.get_width() , fontsize=70)
    my_x_ticks = np.arange(0, 100, 20)
    plt.xticks(my_x_ticks)
    plt.show()
      

In [13]:

check_custom_Detection(['Telemetry','MSSP'])
      

APT29評估廠商的精準檢出統計

對General、Technique、Tactic類型的檢出資料進行統計。

結論：此類數值并不是越大就代表廠商越牛，僅能反應廠商對部分低誤報、低噪點的技術矩陣跟進力度。

In [14]:

check_custom_Detection(['General','Technique','Tactic'])
      

APT29評估廠商按照主要檢測類型的價值進行打分排名

打分标準：

• None 無檢出 0分
• Telemetry 遙測型檢出 0.5分
• MSSP 安全營運型檢出 0.6分
• General 通用型檢出 0.7分
• Tactic 戰術型檢出和Technique 技術型檢出 均為1分

結論：

60分以上的形成第一梯隊，廠商隻有微小差距

60分以下至55分形成第二梯隊，廠商以1分左右形成梯度差距

55分以下的廠商形成第三梯隊，開始完全掉隊

In [15]:

def check_flag_Detection():  
    
    flag_list = []
    flag_data = copy.deepcopy(all_data)
    
    def check_deep1_Detection(DetectionNote):
        Detection = 0
        for xl in DetectionNote:
            if xl['DetectionType']=='None':
                Detection = 0
            elif xl['DetectionType']=='N/A':
                Detection = 0
            elif xl['DetectionType']== 'Telemetry':
                Detection = 0.5
            elif xl['DetectionType']== 'MSSP':
                Detection = 0.6
            elif xl['DetectionType']== 'General':
                Detection = 0.7
            elif xl['DetectionType']== ('Tactic' or 'Technique'):
                Detection = 1
        return Detection

    for i,d in flag_data.items():
        d['Detections'] =  d['Detections'].apply(lambda x: check_deep1_Detection(x)) 
        d_count =d['Detections'].sum()   
        flag_list.append( (i,d_count) )
        
    tel_df = pd.DataFrame( flag_list ).sort_values(1,ascending=True).round(1)
    ax = tel_df.plot(kind='barh',figsize=(110,70),x=0, y=1, fontsize=65)
    for i in ax.patches:
        ax.text(i.get_width(), i.get_y() + 0.2, i.get_width() , fontsize=70)
    my_x_ticks = np.arange(0, 100, 20)
    plt.xticks(my_x_ticks)
    plt.show()
      

In [16]:

check_flag_Detection()
      

https://nbviewer.jupyter.org/github/raystyle/REPORT/blob/master/attck_report.ipynb