天天看點

記修複CMAK不能從Kafka Borke擷取Metrics

使用Docker部署了跨主機的3節點Kafka叢集,網絡使用的是host模式,暴露JMX_PORT端口後,CMAK還是擷取不到Metrics資料,報錯類似如下:

Failed to get broker metrics for BrokerIdentity(1,192.168.4.11,9999,false,true,Map(PLAINTEXT -> 9092))
           

在參考如下資料:

  1. confluent kafka官方文檔有關jmx端口配置

    https://docs.confluent.io/platform/current/installation/docker/operations/monitoring.html#configure-environment

  2. Github 中 yahoo/CMAK 的issue的讨論

    https://github.com/yahoo/CMAK/issues/563

    重要原文如下:

Well, it looks like KafkaManager can't connect to the JMX port... first I would check that it's possible to connect from KM host to each of the brokers using netcat, you should see something like this:

$ netcat -z -n -v 10.0.20.7 9096

Connection to 10.0.20.7 9096 port [tcp/*] succeeded!

In case it's successful, then check which interface is KM using, check firewalls, etc, something is blocking the request between KM and Kafka.

If netcat fails then most likely there is a problem with the JMX configuration in kafka, check -Dcom.sun.management.jmxremote.local.only=false and in hosts with multiple interfaces -Djava.rmi.server.hostname= is configured to the correct ip.

environment:
      - JMX_PORT=9999
      - KAFKA_JMX_OPTS=-Djava.rmi.server.hostname=192.168.4.11 -Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.rmi.port=9999 -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false 
           
environment:
      - JMX_PORT=9999
      - KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote.rmi.port=9999 -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false