zeppelin出現ConnectException: Connection refused解決方法

zeppelin出現ConnectException: Connection refused

問題

一直用zeppelin做分析工作台，最近想把現用的環境整體打包複制到另一個叢集，在新的叢集啟動spark interpreter時出現了

ConnectException

，詳細出錯資訊如下：

ERROR [2018-03-05 17:33:01,109] ({Thread-31} RemoteInterpreterEventPoller.java[run]:77) - Can't get RemoteInterpreterEvent
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
        at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:)
        at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:)
        at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:)
        at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:)
        at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:)
Caused by: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused)
        at org.apache.thrift.transport.TSocket.open(TSocket.java:)
        at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:)
        ...  more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:)
        at java.net.Socket.connect(Socket.java:)
        at org.apache.thrift.transport.TSocket.open(TSocket.java:)
        ...  more
           

解決方法

老的zeppelin環境下，在

conf/zeppelin-env.sh

下配置了

export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.4.0"

， 在spark job送出時需要從遠端倉庫下載下傳指定的packages包，這裡的遠端倉庫也可能是私有的配置，是以利用以下方法解決：

• 去掉zeppelin-env.sh中的submit option中的packages配置

  export SPARK_SUBMIT_OPTIONS="--packages com.databricks:spark-csv_2.10:1.4.0"

  如果嘗試spark interpreter是否可用，不行再去掉repo
• 去掉interpreter中的Repositories私有配置