https://discuss.elastic.co/t/es-vs-lucene-memory/20959
I've read the recommendations for ES_HEAP_SIZE
which
basically state to set -Xms and -Xmx to 50% physical RAM.
It says the rest should be left for Lucene to use (OS filesystem caching).
But I'm confused on how Lucene uses that. Doesn't Lucene run in the same
JVM as ES? So they would share the same max heap setting of 50%.
nik9000 Nik Everett Elastic Team Member
Nov '14
Lucene runs in the same JVM as Elasticsearch but (by default) it mmaps
files and then iterates over their content inteligently. That means most
of its actual storage is "off heap" (its a java buzz-phrase). Anyway,
Linux will serve reads from mmaped files from its page cache. That is why
you want to leave linux a whole bunch of unused memory.
Nik
就是官方人員建議預留一些空閑的記憶體給ES(lucene)的底層檔案系統用于File cache。
pountz Adrien Grand Elastic Team Member
Indeed the behaviour is the same on Windows and Linux: memory that is not
used by processes is used by the operating system in order to cache the
hottest parts of the file system. The reason why the docs say that the rest
should be left to Lucene is that most disk accesses that elasticsearch
performs are done through Lucene.
I used procexp and VMMap to double check, ya, i think they are file system
cache.
Is there anyway to control the size of file system cache? Coz now it's
easily driving up OS memory consumption. When it's reaching 100%, the node
would fail to respond...
他們也遇到ES機器的記憶體（java heap+檔案系統）使用達到了100%。
不過該文章沒有給出解決方案。
類似ES記憶體問題在：https://discuss.elastic.co/t/jvm-memory-increase-1-time-more-xmx16g-while-elasticsearch-heap-is-stable/55413/4
其操作是每天都定期查詢。
Elasticsearch uses not only heap but also out-of-heap memory buffers because of Lucene.
I just read the Lucene blog post and I already know that Lucene/ES start to use the file system cache (with MMapDirectory).
That why in my graph memory you can see: Free (in green) + Used memory (in red) + cached memory (the FS cache in blue).
https://discuss.elastic.co/t/memory-usage-of-the-machine-with-es-is-continuously-increasing/23537/2
提到：ES記憶體每天都上升200MB，而重新開機ES則一切又正常了。
Note
that when I restart the ES it gets cleared(most of it, may be OS clears up
this cache once it sees that the parent process has been stopped).
When the underlying lucene engine interacts with a segment the OS will
leverage free system RAM and keep that segment in memory. However
Elasticsearch/lucene has no way to control of OS level caches.
這個是作業系統的cache，ES本身無法控制。
轉自：http://farll.com/2016/10/high-memory-usage-alarm/
Linux Centos伺服器記憶體使用率過高的報警, 最後得出結論是因為 nss-softokn的bug導緻curl 大量請求後, dentry 緩存飙升.
問題的開始是收到雲平台發過來的記憶體使用率平均值超過報警值的短信, 登入雲監控背景檢視發現從前兩天開始記憶體使用曲線緩慢地呈非常有規律上升趨勢.
用top指令, 然後使用M按記憶體使用大小排序發現并沒有特别消耗記憶體的程序在跑, 用H查詢線程情況也正常. 最高的mysql占用6.9%記憶體. memcached程序telnet後stats并未發現有記憶體消耗過大等異常情況.
free -m 檢視 -/+ buffers/cache used很高, 可用的free記憶體隻剩下百分之十幾, 那麼記憶體消耗究其在哪裡去了?
使用cat /proc/meminfo 檢視記憶體配置設定更詳細的資訊: Slab 和 SReclaimable 達幾個G, Slab記憶體配置設定管理, SReclaimable看字面意思是可回收記憶體.
因為最近并沒有對伺服器配置和程式代碼進行變更, 是以第一反應是想是不是AliyunUpdate自動更新程式更改了什麼. 但是 strace 依次監控Ali系列的程序并沒有發現有大量的檔案操作. crond也發現會讀寫少量的session等臨時檔案, 但數量少并不至于到這個程度.
官方的工單回複是可以考慮更新伺服器記憶體，如果記憶體不足影響業務，需要臨時釋放一下slab占用的記憶體， 參考以下步驟：
隻給了個臨時解決方案, 總不能還要開個自動任務定時執行回收dentry cache的任務吧. 不過我們逐漸接近了事情的本質, 即目前看來是過高的slab記憶體無法回收導緻記憶體不足. 那麼我們可以提高slab記憶體釋放的優先級, Linux 提供了 vfs_cache_pressure 這個參數, 預設為100, 設定為高于100的數, 數值越大slab回收優先級越高(root 身份運作):
并不建議通過如上Laravel隊列文章最後提到的使用min_free_kbytes 和 extra_free_kbytes參數來實作回收slab緩存的, 核心文檔中比較明确地指明了vfs_cache_pressure 适合于控制directory 和inode 緩存的回收:
雖然slab占用過多記憶體得到了有效控制和實時回收, 但實際上我們還是沒有找到問題的本質, 按理來說預設的vfs_cache_pressure值應該是能夠較好地實作可用記憶體和slab緩存之間的平衡的.
最後去查該時間節點的網站服務日志才發現, 記憶體使用率一直升的原因在于從這個時間開始, 有系統crontab任務一直在不斷地通過curl請求外部資源api接口, 而他這個api基于https. 罪魁禍首就在于Libcurl附帶的Mozilla網絡安全服務庫NSS(Network Security Services) 的bug, 隻有curl請求ssl即https的資源才會引入NSS.
第一步: 確定nss-softokn是已經解決了bug的版本
檢視是否大于等于3.16.0版本(根據上面的bug修複連結, 大于nss-softokn-3.14.3-12.el6 也可以):
<code>yum list nss-softokn</code>
若版本太低需要更新:
<code>sudo yum update -y nss-softokn</code>
第二步: 設定變量NSS_SDB_USE_CACHE=yes
第三步: 重新開機Web伺服器
重新開機Apache 或 重新開機Nginx和php-fpm. 否則curl_error可能出現 <code>Problem with the SSL CA cert (path? access rights?)</code> 的錯誤.
設定後dentry cache終于慢慢恢複正常.
本文轉自張昺華-sky部落格園部落格，原文連結：http://www.cnblogs.com/bonelee/p/8066544.html，如需轉載請自行聯系原作者