天天看点

CDH5.2挂载fuse-dfs问题解决

问题描述:

要在本地挂载hdfs文件系统,然后在官网找到一下中文文档,详细内容连接如下:

http://www.cloudera.com/content/cloudera/zh-CN/documentation/core/v5-3-x/topics/cdh_ig_hdfs_mountable.html

需要用到的命令:

非HA:

$ mkdir -p <mount_point>
$ hadoop-fuse-dfsdfs://<name_node_hostname>:<namenode_port> <mount_point>
           

HA:

$ mkdir -p <mount_point>
$ hadoop-fuse-dfs dfs://<nameservice_id> <mount_point>
           
但是挂载时出现问题:信息如下(找不到共享库中的文件):
/opt/cloudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/bin/../lib/hadoop/bin/fuse_dfs: error while loading shared libraries: libjvm.so: cannot open shared object file: No such file or directory
           

解决:

经过排查发现是环境变量配置问题 该环境变量主要用于指定查找共享库(动态链接库)时除了默认路径之外的其他路径. 。(该路径在默认路径之前查找) hadoop-fuse-dfs命令代码片段
if [ "${LD_LIBRARY_PATH}" = "" ]; then
  export LD_LIBRARY_PATH=/usr/lib
  for f in `find ${JAVA_HOME}/ -name client -prune -o -name libjvm.so -exec dirname {} \;`; do
    export LD_LIBRARY_PATH=$f:${LD_LIBRARY_PATH}
    export LD_LIBRARY_PATH="${BIN_DIR}/../lib64":${LD_LIBRARY_PATH}
  done
fi
           
如果环境变量LD_LIBRARY_PATH为空,那经过上面执行变为:LD_LIBRARY_PATH=/opt/c loudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/bin/../lib64:/home/pub/jdk/jre/lib/amd64/server 但是因为本地/etc/profile文件中:
export  LD_LIBRARY_PATH=/usr/lib:/oracle/app/product/11.2.0/db_1/lib/:/oracle/app/product/11.2.0/db_1/lib32/:/lib/
           
环境变量LD_LIBRARY_PATH不为空并且不包含fuse的共享库,所以在本地/etc/profile文件中把fuse共享库与java共享库加上去就可以了,如下:
export  LD_LIBRARY_PATH=/usr/lib:/oracle/app/product/11.2.0/db_1/lib/:/oracle/app/product/11.2.0/db_1/lib32/:/lib/:/opt/c
loudera/parcels/CDH-5.2.1-1.cdh5.2.1.p0.12/bin/../lib64:/home/pub/jdk/jre/lib/amd64/server
           
最后df –lh看到已经挂载上去了 fuse_dfs              xxT  xxT   xxT   58% /hdfs

继续阅读