HugePages on Linux
HugePages是linux核心的一個特性,使用hugepage可以用更大的記憶體頁來取代傳統的4K頁面。使用HugePage主要帶來如下好處
1,沒有swap。Notswappable: HugePages are not swappable. Therefore there is no page-in/page-outmechanism overhead.HugePages are universally regarded as pinned.
2,減輕快表壓力。Reliefof TLB pressure:TLB表格的更小了,效率提高
3,減輕換頁表的負載。每個表單需64位元組,如果管理50GB的實體記憶體,如果使用傳統4K頁面pagetable需要800M大小,而是用HugePages僅需要40M
4,提高記憶體的性能,降低CPU負載,原理同上
HugePages和oracle AMM(自動記憶體管理)是互斥的,所有使用HugePages必須設定記憶體參數MEMORY_TARGET / MEMORY_MAX_TARGET 為0
配置HugePages的具體步驟
1、修改核心參數memlock,機關是KB,如果記憶體是128G,memlock的大小要稍微小于實體記憶體。計劃lock 100GB的記憶體大小。參數設定為大約SGA是沒有壞處的。
vi /etc/security/limits.conf
*soft memlock 104857600
*hard memlock 104857600
2,使用資料庫帳号驗證大小
[[email protected] ~]$ ulimit -a|grep lock
core file size (blocks, -c) 0
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) 104857600
file locks (-x) unlimited
3,如果使用AMM記憶體管理,要取消改設定。MEMORY_TARGET和 MEMORY_MAX_TARGET參數設定為0
SQL> alter system reset memory_targetscope=spfile ;
SQL> alter system resetmemory_max_target scope=spfile;
SQL> alter system set sga_target = 10Gscope=spfile;
SQL> alter system setpga_aggregate_target = 4G scope = spfile;
4,計算需要使用的hugepage頁面的大小。hugepage目前隻能用于共享記憶體段等少量記憶體類型,例如oracle SGA。PGA則不适用,這些記憶體一般不能用于其它用途,是以設定太小則不足夠放下所有記憶體段,太大則空間浪費。
簡單的計算原理是total SGA_MAX_SIZE(多個instance的總和)/hugepagesize + N
N為少量記憶體盈餘,一般多出100就足夠了。如果主機記憶體128GB,計劃70GB用于SGA共享記憶體,則大記憶體頁需70×1024/2=35840
也可使用oracle提供的計算公式,基本原理是使用ipcs -m來計算共享記憶體段的大小。統計前注意關閉AMM;
vi hugepages_settings.sh
#!/bin/bash # # hugepages_settings.sh # # Linux bash script to compute values for the # recommended HugePages/HugeTLB configuration # # Note: This script does calculation for all shared memory # segments available when the script is run, no matter it # is an Oracle RDBMS shared memory segment or not. # # This script is provided by Doc ID 401749.1 from My Oracle Support # http://support.oracle.com # Welcome text echo " This script is provided by Doc ID 401749.1 from My Oracle Support (http://support.oracle.com) where it is intended to compute values for the recommended HugePages/HugeTLB configuration for the current shared memory segments. Before proceeding with the execution please make sure that: * Oracle Database instance(s) are up and running * Oracle Database 11g Automatic Memory Management (AMM) is not setup (See Doc ID 749851.1) * The shared memory segments can be listed by command: # ipcs -m Press Enter to proceed..." read # Check for the kernel version KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'` # Find out the HugePage size HPG_SZ=`grep Hugepagesize /proc/meminfo | awk '{print $2}'` # Initialize the counter NUM_PG=0 # Cumulative number of pages required to handle the running shared memory segments for SEG_BYTES in `ipcs -m | awk '{print $5}' | grep "[0-9][0-9]*"` do MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q` if [ $MIN_PG -gt 0 ]; then NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q` fi done RES_BYTES=`echo "$NUM_PG * $HPG_SZ * 1024" | bc -q` # An SGA less than 100MB does not make sense # Bail out if that is the case if [ $RES_BYTES -lt 100000000 ]; then echo "***********" echo "** ERROR **" echo "***********" echo "Sorry! There are not enough total of shared memory segments allocated for HugePages configuration. HugePages can only be used for shared memory segments that you can list by command: # ipcs -m of a size that can match an Oracle Database SGA. Please make sure that: * Oracle Database instance is up and running * Oracle Database 11g Automatic Memory Management (AMM) is not configured" exit 1 fi # Finish with results case $KERN in '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`; echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;; '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;; *) echo "Unrecognized kernel version $KERN. Exiting." ;; esac # End |
5,修改vm.nr_hugepages參數,值為上步計算的數值
vi /etc/sysctl.conf
vm.nr_hugepages = 1496
sysctl -p 指令使配置生效。
6,關閉資料庫,重新開機主機和資料庫(理論上不需要重新開機主機,建議重新開機)
7,驗證是否設定正确
grep HugePages /proc/meminfo
HugePages_Free小于HugePages_Total的值則表示設定成功。如果HugePages_Rsvd應該保持少量保留記憶體。
注意,HugePages如果配置不恰當會引起系統性能下降等風險,需要慎重。
參考資料
HugePages on Linux: What It Is... and WhatIt Is Not... [ID 361323.1]
HugePages on Oracle Linux 64-bit [ID361468.1]