天天看點
傳回

Python程式設計：RabbitMQ消息隊列

不同程式之間通訊

1.socket

2.disk硬碟檔案

3.broker中間代理

python中：

threading Queue 線程之間通訊，不能跨程序

multiprocessing Queue 父程序與子程序進行互動，或同一個父程序下的多個子程序

RabbitMQ 消息隊列

MQ全稱為Message Queue，一種應用程式對應用程式的通信方法

RabbitMQ 官方位址：

http://www.rabbitmq.com/

安裝：

erlang

http://www.erlang.org/ rabbitmq

參考：

《Windows下RabbitMQ安裝及入門》

http://blog.csdn.net/hzw19920329/article/details/53156015

啟動服務（管理者模式）：rabbitmq-service start 或者：services.msc

通路管理背景：

http://localhost:15672 使用者名：guest，密碼：guest

檢視隊列:rabbitmqctl list_queues

檢視狀态：rabbitmqctl status

rabbitMQ輪詢分發

将消息依次分發給每個消費者

生産者 –> 隊列 –> 消費者1， 消費者2， 消費者3

rabbitMQ消息持久化

伺服器關閉，隊列消失，可以設定持久化隊列名，持久化消息

消費者控制接收數量

消費者可以按需擷取資訊，實作能者多勞

釋出者訂閱者

是即時發送接收

釋出者 –> 交換機 –> 隊列1， 隊列2， 隊列3 –> 訂閱者1， 訂閱者2， 訂閱者3

廣播模式

fanout廣播模式 無選擇接收

direct廣播模式 有選擇接收 隊列綁定關鍵字

topic廣播模式 消息過濾

遇到的問題

問題：無法通路Web管理頁面

解決：

啟動管理子產品：

rabbitmqctl start_app

rabbitmq-plugins enable rabbitmq_management

rabbitmqctl stop

《RabbitMQ無法通路Web管理頁面》

http://blog.csdn.net/u011642663/article/details/54691788

問題：Error: unable to perform an operation

C:\Windows\System32\config\systemprofile.erlang.cookie

拷貝.erlang.cookie檔案覆寫

C:\User\username.erlang.cookie

《Authentication failed》

http://blog.csdn.net/j_shine/article/details/78833456

代碼執行個體

安裝第三方庫：

pip install pika

生産者消費者 

# 發送端

import pika  

queue_name = "hello2" # 隊列名稱

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

# 聲明queue
channel.queue_declare(queue=queue_name)  # durable=True消息名稱持久化

# 發送消息，需要通過路由器
channel.basic_publish(exchange="",
                      routing_key=queue_name,
                      body="hello, world!",
                      # 消息持久化 make message persistent
                      #properties=pika.BasicProperties(delivery_mode=2)
                      )
print("send hello")
connection.close()

# 持久化之後，伺服器重新開機不消失      
# 接收端

import pika
import time

queue_name = "hello2" # 隊列名稱

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

# 不知道用戶端還是服務端先啟動，為了確定這個隊列存在，兩端都需要聲明
channel.queue_declare(queue=queue_name)

channel.basic_qos(prefetch_count=1)  # 最多處理一個資訊，處理完再接收

def callback(ch, method, properties, body): # 回調函數
    print("ch:", ch)
    print("method:", method)
    print("properties:", properties)
    print("接收到資訊：", body)
    time.sleep(30)
    # ch.basic_ask(delivery_tag=method.delivery_tag)  # 消息确認

channel.basic_consume(callback,  # 如果收到消息就調用函數處理消息
                      queue=queue_name,
                      no_ack=True)  #acknowledgement确認

print("waiting for message, ctrl+c break")

# 啟動接收
channel.start_consuming()

1對多的發送廣播

廣播模式：fanout

# 釋出者

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

channel.exchange_declare(exchange="logs", exchange_type="fanout")

message = "hello world"

channel.basic_publish(exchange="logs",
                      routing_key="",
                      body=message)

print("send ok")

connection.close()      
# 訂閱者

import pika

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

channel.exchange_declare(exchange="logs", exchange_type="fanout")

result = channel.queue_declare(exclusive=True)  # 随機配置設定隊列名

queue_name = result.method.queue

channel.queue_bind(exchange="logs", queue=queue_name)
print("waiting for logs")

def callback(ch, method, properties, body):
    print("body:", body)

channel.basic_consume(callback, queue=queue_name, no_ack=True)

channel.start_consuming()

消息過濾

有選擇的接收消息，廣播模式：direct

# 實作1對多的發送

# 釋出者
import pika
import sys

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

channel.exchange_declare(exchange="direct_logs", exchange_type="direct")

severity = sys.argv[1] if len(sys.argv)>1 else "info"  # severity嚴重程度
message = " ".join(sys.argv[:2] or "hello world!")


channel.basic_publish(exchange="direct_logs",
                      routing_key=severity,
                      body=message)

print("send ok")

connection.close()
      
# 訂閱者


import pika
import sys

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

channel.exchange_declare(exchange="direct_logs", exchange_type="direct")

result = channel.queue_declare(exclusive=True)  # 随機配置設定隊列名

queue_name = result.method.queue

# 擷取參數
severities = sys.argv[1:]
if not severities:
    sys.stderr.write("usage: %s [info] [warning] [error]" % sys.argv[0])
    sys.exit(1)

for severity in severities:
    channel.queue_bind(exchange="direct_logs",
                       routing_key=severity,
                       queue=queue_name)

print("waiting...")

def callback(ch, method, properties, body):
    print("routing_key:", method.routing_key)
    print("body:", body)

channel.basic_consume(callback, queue=queue_name, no_ack=True)

channel.start_consuming()

通配符消息過濾

表達式符号說明：#代表一個或多個字元，*代表任何字元

廣播模式：topic

# 實作1對多的發送

# 釋出者
import pika
import sys

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

channel.exchange_declare(exchange="topic_logs", exchange_type="topic")

severity = sys.argv[1] if len(sys.argv)>1 else "info"  # severity嚴重程度
message = " ".join(sys.argv[:2] or "hello world!")


channel.basic_publish(exchange="topic_logs",
                      routing_key=severity,
                      body=message)

print("send ok")

connection.close()
      
# 訂閱者

import pika
import sys

connection = pika.BlockingConnection(
    pika.ConnectionParameters("localhost")
)

channel = connection.channel()

channel.exchange_declare(exchange="topic_logs", exchange_type="topic")

result = channel.queue_declare(exclusive=True)  # 随機配置設定隊列名

queue_name = result.method.queue

# 擷取參數
severities = sys.argv[1:]
if not severities:
    sys.stderr.write("usage: %s [info] [warning] [error]" % sys.argv[0])
    sys.exit(1)

for severity in severities:
    channel.queue_bind(exchange="topic_logs",
                       routing_key=severity,
                       queue=queue_name)

print("waiting...")

def callback(ch, method, properties, body):
    print("routing_key:", method.routing_key)
    print("body:", body)

channel.basic_consume(callback, queue=queue_name, no_ack=True)

channel.start_consuming()

Remote procedure call (RPC)

遠端程式調用

# 伺服器端

import pika
import time

connection = pika.BlockingConnection(
    pika.ConnectionParameters(host='localhost')
)

channel = connection.channel()

channel.queue_declare(queue='rpc_queue')


def fib(n):  # 被調用函數
    if n == 0:
        return 0
    elif n == 1:
        return 1
    else:
        return fib(n - 1) + fib(n - 2)


def on_request(ch, method, props, body):
    n = int(body)

    print(" [.] fib(%s)" % n)
    response = fib(n)

    ch.basic_publish(exchange='',
                     routing_key=props.reply_to,
                     properties=pika.BasicProperties(
                         correlation_id= props.correlation_id
                     ),
                     body=str(response))
    ch.basic_ack(delivery_tag=method.delivery_tag)


channel.basic_qos(prefetch_count=1)
channel.basic_consume(on_request, queue='rpc_queue')

print(" [x] Awaiting RPC requests")
channel.start_consuming()      
# 用戶端

import pika
import uuid


class FibonacciRpcClient(object):
    def __init__(self):
        self.connection = pika.BlockingConnection(
            pika.ConnectionParameters(host='localhost')
        )

        self.channel = self.connection.channel()

        # 生成随機隊列
        result = self.channel.queue_declare(exclusive=True)
        self.callback_queue = result.method.queue

        self.channel.basic_consume(self.on_response,
                                   no_ack=True,
                                   queue=self.callback_queue
                                   )

    def on_response(self, ch, method, props, body):
        if self.corr_id == props.correlation_id:
            self.response = body

    def call(self, n):
        self.response = None
        self.corr_id = str(uuid.uuid4())  # 用于區分發送指令和接收結果的一緻性
        self.channel.basic_publish(exchange='',
                                   routing_key='rpc_queue',
                                   properties=pika.BasicProperties(
                                       reply_to=self.callback_queue,
                                       correlation_id=self.corr_id,
                                   ),
                                   body=str(n))
        while self.response is None:
            self.connection.process_data_events()
        return int(self.response)


fibonacci_rpc = FibonacciRpcClient()

print(" [x] Requesting fib(30)")
response = fibonacci_rpc.call(30)
print(" [.] Got %r" % response)

參考文章：

《Python之路,Day9 - 異步IO\資料庫\隊列\緩存》

http://www.cnblogs.com/alex3714/articles/5248247.html
消息中間件 緩存 資料庫 資料安全/隐私保護 Python Windows python rabbitmq消息 RabbitMQ消息隊列 rabbitmq消息隊列分發機制 rabbitmq消息隊列工作隊列 rabbitmq使用消息隊列
上一篇: Python程式設計：協程coroutine手動切換協程自動切換協程gevent 多并發socketselect 多并發socketselectors 多并發socket異步爬蟲(并行)
下一篇: SQL：redis緩存資料庫深入簡介配置檔案

繼續閱讀