【shell資源限制】RLIMIT_MEMLOCK too small

在使用MPI時遇到了資源受限的報錯

RLIMIT_MEMLOCK too small

這是由于shell預設的資源不足造成的

可以使用

ulimite

指令解決

1.Ulimite

遇到

RLIMIT_MEMLOCK too small

報錯主要是由于預設設定的資源不足以完成指令，特别是在MPI運作多個線程時會出現這樣的問題，根據提示來修改目前shell所需要的資源大小。首先可以利用

ulimite -a

檢視目前預設配置設定的資源：

>>> ulimit -a
>>>
ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7220
max locked memory       (kbytes, -l) 64    <<<<<這裡就是報錯的位置
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1000000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7220
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
           

我們看到報錯的位置locked memory隻有64k，需要放大這個資源限制：

# 直接設成不受限
ulimit -l unlimited
# 再檢視結果
>>> ulimit -a
>>>

core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 772787
max locked memory       (kbytes, -l) unlimited    <<<<<<已經修改成不受限
max memory size         (kbytes, -m) unlimited 
open files                      (-n) 1000000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 772787
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
           

2.shell資源管理

此外可以利用

ulimit -h

檢視更多使用請看

> ulimit --help  
> >>
Syntax
      ulimit [-abcdefHilmnpqrsStTuvx] [limit]

Key
   -S   Set a soft limit for the given resource.
   -H   Set a hard limit for the given resource.

   -a   All current limits are reported.
   -b   The maximum socket buffer size.
   -c   The maximum size of core files created. 
   -d   The maximum size of a process's data segment.
   -e   The maximum scheduling priority ("nice") 
   -f   The maximum size of files created by the shell(default option).
   -i   The maximum number of pending signals.
   -l   The maximum size that can be locked into memory. 
   -m   The maximum resident set size. 
   -n   The maximum number of open file descriptors. 
   -p   The pipe buffer size.
   -q   The maximum number of bytes in POSIX message queues.
   -r   The maximum real-time scheduling priority.
   -s   The maximum stack size. 
   -t   The maximum amount of cpu time in seconds.
   -T   The maximum number of threads.
   -u   The maximum number of processes available to a single user.
   -v   The maximum amount of virtual memory available to the process.
   -x   The maximum number of file locks.
   # from:https://ss64.com/bash/ulimit.html
           

ref:ulimit, 詳解1, 詳解2，原始解決方法，修改core檔案大小, shell詳細資源管理，detail，其他資源管理方法

pic from pexels.com