天天看點

pyspark kafka createDirectStream和createStream 差別

Kafka的程序ID為9300,占用端口為9092

QuorumPeerMain為對應的zookeeper執行個體,程序ID為6379,在2181端口監聽

是以在運作官方例子時候

一個是

./bin/spark-submit --jars ~/spark-streaming-kafka-0-8-assembly_2.11-2.2.0.jar examples/src/main/python/streaming/direct_kafka_wordcount.py localhost:9092 test

另外一個是:

 ./bin/spark-submit --jars ~/spark-streaming-kafka-0-8-assembly_2.11-2.2.0.jar examples/src/main/python/streaming/direct_kafka_wordcount.py localhost:2181 test

參考:

https://spark.apache.org/docs/1.6.1/streaming-kafka-integration.html

http://zhangfengzhe.blog.51cto.com/8855103/1556650

本文轉自張昺華-sky部落格園部落格,原文連結:http://www.cnblogs.com/bonelee/p/7443026.html,如需轉載請自行聯系原作者