【shell资源限制】RLIMIT_MEMLOCK too small

在使用MPI时遇到了资源受限的报错

RLIMIT_MEMLOCK too small

这是由于shell默认的资源不足造成的

可以使用

ulimite

命令解决

1.Ulimite

遇到

RLIMIT_MEMLOCK too small

报错主要是由于默认设置的资源不足以完成指令，特别是在MPI运行多个线程时会出现这样的问题，根据提示来修改当前shell所需要的资源大小。首先可以利用

ulimite -a

查看当前默认分配的资源：

>>> ulimit -a
>>>
ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7220
max locked memory       (kbytes, -l) 64    <<<<<这里就是报错的位置
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1000000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7220
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
           

我们看到报错的位置locked memory只有64k，需要放大这个资源限制：

# 直接设成不受限
ulimit -l unlimited
# 再查看结果
>>> ulimit -a
>>>

core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 772787
max locked memory       (kbytes, -l) unlimited    <<<<<<已经修改成不受限
max memory size         (kbytes, -m) unlimited 
open files                      (-n) 1000000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 772787
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
           

2.shell资源管理

此外可以利用

ulimit -h

查看更多使用请看

> ulimit --help  
> >>
Syntax
      ulimit [-abcdefHilmnpqrsStTuvx] [limit]

Key
   -S   Set a soft limit for the given resource.
   -H   Set a hard limit for the given resource.

   -a   All current limits are reported.
   -b   The maximum socket buffer size.
   -c   The maximum size of core files created. 
   -d   The maximum size of a process's data segment.
   -e   The maximum scheduling priority ("nice") 
   -f   The maximum size of files created by the shell(default option).
   -i   The maximum number of pending signals.
   -l   The maximum size that can be locked into memory. 
   -m   The maximum resident set size. 
   -n   The maximum number of open file descriptors. 
   -p   The pipe buffer size.
   -q   The maximum number of bytes in POSIX message queues.
   -r   The maximum real-time scheduling priority.
   -s   The maximum stack size. 
   -t   The maximum amount of cpu time in seconds.
   -T   The maximum number of threads.
   -u   The maximum number of processes available to a single user.
   -v   The maximum amount of virtual memory available to the process.
   -x   The maximum number of file locks.
   # from:https://ss64.com/bash/ulimit.html
           

ref:ulimit, 详解1, 详解2，原始解决方法，修改core文件大小, shell详细资源管理，detail，其他资源管理方法

pic from pexels.com