CentOS/RedHat install Tensorflow in HPC

Problem

Tensorflow(a .so library file in the python library path) need the Glibc2.15 + , but the Glibc version in RedHat in HPC is 2.12. This make the tensorflow install fail.

Analysis(Principle)

The .so dynamic library file is linked/loaded to the python executable file by the

Dynamic linker/loader

in

runtime

. We can specify the new Glibc’s library file(

libc.so.6

) path to the

LD_LIBRARY_PATH

environment, but the

Glibc

and the

Dynamic loader/ld.so.2

is corresponding. Thus we also need to specify the new

Glibc

’s corresponding

ld.so.2

file.

Refer to:

http://man7.org/linux/man-pages/man8/ld.so.8.html

http://man7.org/linux/man-pages/man1/ldd.1.html

https://www.cs.virginia.edu/~dww4s/articles/ld_linux.html

https://unix.stackexchange.com/questions/122670/using-alternate-libc-with-ld-linux-so-hacks-cleaner-method

Answer

We explicit specify the

ld.so.2

’s path in command line instead of the default

ld.so.2

file path which specified in the binary file1 2. And use the

--library-path

parameter of

ld.so.2

instead of

LD_LIBRARY_PATH

environment.3 For example:

${HOME}/local/lib64/ld-linux-x86-64.so.2 --library-path ${HOME}/local/lib64:${HOME}/local/usr/lib64 ${HOME}/anaconda/tensorflow/bin/python
           

Refer to:

http://xr0038.hatenadiary.jp/entry/2016/08/31/165011

https://stackoverflow.com/questions/33655731/error-while-importing-tensorflow-in-python2-7-in-ubuntu-12-04-glibc-2-17-not-f/34897674#34897674

Set up the environment

modified from4

For the previous steps, please refer to the original blog.

# activate the tensorflow pyenv in the anaconda
source activate tensorflow
LDPATH="${HOME}/local/lib64/ld-linux-x86-64.so.2"
LDLIBS="${HOME}/local/lib64"
LDLIBS="${LDLIBS}:${HOME}/local/usr/lib64"
LDLIBS="${LDLIBS}:/opt/cuda-7.5/lib64"
PYPATH="${HOME}/anaconda/tensorflow/bin/python"

# ${LDPATH} --library-path ${LDLIBS} ${PYPATH}

# ipython は以下のように起動する．
# IPYPATH="${HOME}/.pyenv/versions/TensorFlow/bin/ipython"

# ${LDPATH} --library-path ${LDLIBS} ${PYPATH} ${IPYPATH}

alias python-tensorflow="${LDPATH} --library-path ${LDLIBS} ${PYPATH}"
# alias ipython-tensoflow="${LDPATH} --library-path ${LDLIBS} ${PYPATH} ${IPYPATH}"
           

Use

~ python-tensorflow
>>> import tensorflow as tf
>>> sess = tf.Session()
>>> exit()

~ python-tensorflow xxx.py
           

1. http://man7.org/linux/man-pages/man8/ld.so.8.html ↩
2. patchELF ↩
3. http://man7.org/linux/man-pages/man8/ld.so.8.html#OPTIONS ↩
4. http://xr0038.hatenadiary.jp/entry/2016/08/31/165011 ↩