Tornado - Middleware

目錄

一、RequestHandler 基礎類支援

RequestHandler.initialize() - 用于子類初始化，每個請求都會調用

RequestHandler.prepare() - 處理請求方法前的調用函數

RequestHandler.on_finish() - 請求結束後的調用函數

RequestHandler.finish（chunk：Union [str，bytes，dict] = None ） - 結束 HTTP 請求，傳回 future 對象

RequestHandler.write_error（status_code：int， ** kwargs） - 捕獲錯誤

二、Middleware - 基于HTTP實作

2-0 基本思路總結

2-1 settings 配置

2-2 run.py

2-3 Middleware 相關基類

2-3-1 中間件基類寫法 - 不重寫倆個方法則報錯

2-4 繼承處理類

三、Middleware - 基于 WS 實作

3-0 基礎支援方法

3-1 基于繼承包裹

3-1-0 settings 全局配置

3-1-1 基類

3-1-2 中間件

3-1-3 run.py

3-2 基于繼承中間件實作 beat-ping&pong

四、Django Middleware 源碼流程解析

4-1 檢視Django流程入口，擷取資訊

4-2 BaseHandler 分析中間件加載過程

4-2 middle load 源碼解析

一、RequestHandler 基礎類支援

RequestHandler.initialize() - 用于子類初始化，每個請求都會調用

RequestHandler.prepare() - 處理請求方法前的調用函數

RequestHandler.on_finish() - 請求結束後的調用函數

RequestHandler.finish（chunk：Union [str，bytes，dict] = None ） - 結束 HTTP 請求，傳回 future 對象

RequestHandler.write_error（status_code：int， ** kwargs） - 捕獲錯誤

二、Middleware - 基于HTTP實作

參考學習

2-0 基本思路總結

1. 自定義 middleware 清單
2. MiddleHandler 類内的 initialize() ，初始化屬性，可 dict(middleware=middleware_list) 形式傳參
3. 中間件對象繼承于 MiddleWare 類，該類重寫process_request或process_response方法
4. MiddleHandler 類内的 prepare() 用于擷取list内對象進行調用，調用對應process_request方法
5. on_finish() 方法進行中間件對象的process_response
6. finish() 方法用于結束HTTP請求

類關系總結

• MiddleWare 類 - 中間件基類
• MiddleHandler 類 - 接口處理類，用于擷取并 MiddleWare 對象

2-1 settings 配置

MIDDLEWARE_LIST = [
    'middles.TestMiddleware',
    'middles.TestMiddleware2',
]
           

2-2 run.py

from tornado import ioloop
from tornado.web import Application
​
from tornado_ws.middle.mymiddle.base_class import MiddleHandler
​
class ProfileHandler(MiddleHandler):
    # 若配置，則使用該配置；不配置，預設使用sttings内的配置
    middleware_list = ['middles.TestMiddleware2']
​
    def get(self):
        print('ProfileHandler - get')
​
​
if __name__ == '__main__':
    application = Application([
        (r"/", ProfileHandler),
    ],
    )
    application.listen(8888)
    ioloop.IOLoop.current().start()
           

2-3 Middleware 相關基類

# ----------------------- 基類初始化 -------------------------------
import importlib
from tornado.web import RequestHandler
from tornado_ws.middle.mymiddle.settings import MIDDLEWARE_LIST
​
class Middleware(object):
    '''
    中間件基類
    '''
​
    def process_request(self, request):
        # 子類不實作則抛錯
        raise NotImplemented
​
    def process_response(self, request, response):
        raise NotImplemented
​
​
class MiddleHandler(RequestHandler):
    '''
    中間件處理基類
    基于多中間件的 request和response 處理順序
     - process_request-1、process_request-2
     - process_response-1、process_response-2
    '''
​
    def initialize(self):
        # 若子類内沒有自定義，則使用settings内的預設配置
        try:
            self.middleware_list
        except AttributeError:
            self.middleware_list = MIDDLEWARE_LIST
    
    def prepare(self):
        for middleware in self.middleware_list:
            mpath, mclass = middleware.rsplit('.', maxsplit=1)
            mod = importlib.import_module(mpath)
            getattr(mod, mclass).process_request(self, self)
            
    def on_finish(self):
        for middleware in self.middleware_list:
            mpath, mclass = middleware.rsplit('.', maxsplit=1)
            mod = importlib.import_module(mpath)
            getattr(mod, mclass).process_response(self, self)
            
    def finish(self, chunk=None):
        super().finish(chunk)
​
    def write_error(self, status_code, **kwargs):
        # 若捕獲錯誤，則發送錯誤資訊給client
        exc_cls, exc_instance, trace = kwargs.get("exc_info")
        if status_code != 200:
            self.set_status(status_code)
            self.write({"msg": str(exc_instance)})
           

​
           

2-3-1 中間件基類寫法 - 不重寫倆個方法則報錯

# Django 内部源碼
class MiddlewareMixin:
    def __init__(self, get_response=None):
        self.get_response = get_response
        super().__init__()
​
    def __call__(self, request):
        response = None
        if hasattr(self, 'process_request'):
            response = self.process_request(request)
        response = response or self.get_response(request)
        if hasattr(self, 'process_response'):
            response = self.process_response(request)
        return response
           

class Middleware(object):
    '''
    中間件基類
    ''' 
    # 繼承該類對象被調用的時候觸發，傳回對象的結果 - 在 MiddleHandler 類内進行處理（目前未處理）
    def __call__(self, request):
        response = None
        if hasattr(self, 'process_request'):
            response = self.process_request(request)
        if hasattr(self, 'process_response'):
            response = self.process_response(request)
        return response
           

2-4 繼承處理類

from tornado_ws.middle.mymiddle.base_class import Middleware
​
​
class TestMiddleware(Middleware):
    def process_request(self, request):
        print('TestMiddleware - request')
        print(request) # <__main__.ProfileHandler object at 0x000001EBF1277A58>
​
    def process_response(self, request):
        print('TestMiddleware - response')
​
​
class TestMiddleware2(Middleware):
    def process_request(self, request):
        print('TestMiddleware2 - request')
​
    def process_response(self, request):
        print('TestMiddleware2 - response')
​
           

三、Middleware - 基于 WS 實作

3-0 基礎支援方法

3-1 基于繼承包裹

3-1-0 settings 全局配置

MIDDLEWARE_LIST = [
    'middles.TestMiddleware'
]
           

3-1-1 基類

import importlib
from tornado.websocket import WebSocketHandler
​
from tornado_ws.middle.middle_ws.settings import MIDDLEWARE_LIST
​
​
class Middleware:
    # 中間件基類
    def process_open(self, ws):
        pass
​
    def process_message(self, ws):
        pass
​
    def process_close(self, ws):
        pass
​
​
class WSMiddle(WebSocketHandler):
    # WS 處理基類，用于進行中間件

    _open_middleware = []
    _message_middleware = []
    _close_middleware = []

    def open(self):
        print("WSMiddle opened")

        self._middle_list_handle()
        for open_func in self._open_middleware:
            open_func(self)
​
    def on_message(self, message):
        print("WSMiddle on_message")
        # 将傳遞資料交于ws對象
        self.message = message
        for msg_func in self._message_middleware:
            msg_func(self)
        self.msg_handle(message)
​
    def msg_handle(self, message):
        pass
​
    def on_close(self):
        print("WSMiddle closed")
        for close_func in self._close_middleware:
            close_func(self)
​

    def _middle_list_handle(self):
        try:
            self.middleware_list
        except AttributeError:
            self.middleware_list = options.MIDDLEWARE_LIST

        for middleware in self.middleware_list:
            mpath, mclass = middleware.rsplit('.', maxsplit=1)
            # 初始化放置使用者資訊的字典，添加對應的類
            if not mclass in user_infos:
                user_infos[mclass] = {}
            mod = importlib.import_module(mpath)
            cla_obj = getattr(mod, mclass)
            mw_instance = cla_obj()
            if hasattr(mw_instance, 'process_open'):
                self._open_middleware.append(mw_instance.process_open)
            if hasattr(mw_instance, 'process_message'):
                self._message_middleware.append(mw_instance.process_message)
            if hasattr(mw_instance, 'process_close'):
                self._close_middleware.append(mw_instance.process_close)
​
    # 允許所有跨域通訊，解決403問題
    def check_origin(self, origin):
        return True
           

3-1-2 中間件

from tornado_ws.middle.middle_ws.base import Middleware


class TestMiddleware(Middleware):
    def process_open(self, ws):
        print('TestMiddleware - open')
        # print(ws) # <__main__.EchoWebSocket object at 0x00000210C66D9D30>

    def process_message(self, ws):
        print('TestMiddleware - message')

    def process_close(self, ws):
        print('TestMiddleware - close')


class TestMiddleware2(Middleware):
    def process_open(self, ws):
        print('TestMiddleware2 - open')

    def process_message(self, ws):
        print('TestMiddleware2 - message')

    def process_close(self, *args, **kwargs):
        print('TestMiddleware2 - close')
           

3-1-3 run.py

from tornado.web import Application
from tornado import ioloop

from tornado_ws.middle.middle_ws.base import WSMiddle


class EchoWebSocket(WSMiddle):
    # 前置優先執行
    middleware_list = ['middles.TestMiddleware2', 'middles.TestMiddleware']

    def msg_handle(self, message):
        print(message)


if __name__ == "__main__":
    application = Application([
        (r"/", EchoWebSocket),
    ],
    )
    application.listen(8888)
    ioloop.IOLoop.current().start()
           

3-2 基于繼承中間件實作 beat-ping&pong

run.py

import gzip
import json
import time

from tornado.web import Application
from tornado import ioloop

from tornado_ws.middle.middle_ws.base import WSMiddle
from tornado_ws.middle.middle_ws.settings import user_times, users


class EchoWebSocket(WSMiddle):
    middleware_list = ['middles.PingMiddleware']

    def msg_handle(self, message):
        print(message)


def beatping():
    print('beatping')
    # 向所有使用者推送ping dict，若沒有 pong 傳回則斷開連接配接
    ping_dic = {'ping': round(time.time() * 1000)}
    ping_json = json.dumps(ping_dic)
    ping_byte = bytes(ping_json, encoding='utf-8')
    ping_gzip = gzip.compress(ping_byte)
    for user in users:
        user.write_message(ping_gzip, binary=True)
        user_times[user]['ping'] = ping_dic['ping']
        if user_times[user]['count'] == 2:
            user.close()
            user_times.pop(user)
        else:
            user_times[user]['count'] += 1


if __name__ == "__main__":
    application = Application([
        (r"/", EchoWebSocket),
    ],
    )
    application.listen(8888)
    ioloop.PeriodicCallback(beatping, 5000).start()
    ioloop.IOLoop.current().start()
           

settings.py

MIDDLEWARE_LIST = [
    'middles.TestMiddleware'
]
user_times = {}
users = set()
           

middles.py

import json

from tornado_ws.middle.middle_ws.base import Middleware
from tornado_ws.middle.middle_ws.settings import user_times


class TestMiddleware(Middleware):
    def process_open(self, ws):
        print('TestMiddleware - open')
        # print(ws) # <__main__.EchoWebSocket object at 0x00000210C66D9D30>

    def process_message(self, ws):
        print('TestMiddleware - message')

    def process_close(self, ws):
        print('TestMiddleware - close')


class PingMiddleware(Middleware):
    def process_open(self, ws):
        print('PingMiddleware - open')
        user_times[ws] = dict(
            count=0,
            ping=None,
        )

    def process_message(self, ws):
        print('PingMiddleware - message')
        msg = json.loads(ws.message)
        user_dic = user_times.get(ws)
        print(user_dic)
        # 若pong滿足條件，即重置對應user_times
        if 'pong' in msg:
            if user_dic and user_dic['ping'] == msg['pong']:
                user_times[self]['count'] = 0

    def process_close(self, *args, **kwargs):
        print('PingMiddleware - close')
        if user_times.get(ws):
            # 若非自然斷開連接配接，則删除字典内資訊
            user_times.pop(ws)
           

四、Django Middleware 源碼流程解析

參考學習 = django源碼分析之請求響應流程

4-1 檢視Django流程入口，擷取資訊

# django/core/handlers/wsgi.py
class WSGIHandler(base.BaseHandler):
    request_class = WSGIRequest

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # 附加元件目配置的中間件
        self.load_middleware()

    def __call__(self, environ, start_response):
        set_script_prefix(get_script_name(environ))
        signals.request_started.send(sender=self.__class__, environ=environ)
        # 根據web伺服器傳入的參數，初始化request請求
        request = self.request_class(environ)
        # Django開始處理請求，生成響應
        response = self.get_response(request)

        response._handler_class = self.__class__

        status = '%d %s' % (response.status_code, response.reason_phrase)
        response_headers = list(response.items())
        for c in response.cookies.values():
            response_headers.append(('Set-Cookie', c.output(header='')))
        # wsgi 協定，調用start_response，然後給web伺服器
        start_response(status, response_headers)
        if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'):
            response = environ['wsgi.file_wrapper'](response.file_to_stream)
        return response
           

由上可知，接下來需檢視 base.BaseHander 父類擷取詳細的流程。

4-2 BaseHandler 分析中間件加載過程

4-2 middle load 源碼解析

class BaseHandler:
    _view_middleware = None
    _template_response_middleware = None
    _exception_middleware = None
    _middleware_chain = None

    def load_middleware(self):
        """
        Populate middleware lists from settings.MIDDLEWARE.

        Must be called after the environment is fixed (see __call__ in subclasses).
        """
        self._view_middleware = []
        self._template_response_middleware = []
        self._exception_middleware = []

        # 将self._get_response包裝在異常處理内，此時的handler是一個包裝了_get_response的修飾器函數，進行了異常處理。
        handler = convert_exception_to_response(self._get_response)
        # 周遊settings配置檔案中的中間件
        # 目前擷取反轉了順序，從下至上
        for middleware_path in reversed(settings.MIDDLEWARE):
            # 使用 importlib，将路徑轉變成對象，該函數傳回可調用類對象
            # return getattr(module, class_name)
            middleware = import_string(middleware_path)
            try:
               # 執行個體化middleware對象
                mw_instance = middleware(handler)
            except MiddlewareNotUsed as exc:
                if settings.DEBUG:
                    if str(exc):
                        logger.debug('MiddlewareNotUsed(%r): %s', middleware_path, exc)
                    else:
                        logger.debug('MiddlewareNotUsed: %r', middleware_path)
                continue

            if mw_instance is None:
                raise ImproperlyConfigured(
                    'Middleware factory %s returned None.' % middleware_path
                )
            
            # 隻對_view_middleware，_template_response_middleware，_exception_middleware這3類中間件的連結清單進行了處理。
            # 将預設執行的函數放在函數清單内，後續執行循環調用
            if hasattr(mw_instance, 'process_view'):
                self._view_middleware.insert(0, mw_instance.process_view)
            if hasattr(mw_instance, 'process_template_response'):
                self._template_response_middleware.append(mw_instance.process_template_response)
            if hasattr(mw_instance, 'process_exception'):
                self._exception_middleware.append(mw_instance.process_exception)

            handler = convert_exception_to_response(mw_instance)

        # We only assign to this when initialization is complete as it is used
        # as a flag for initialization being complete.
        self._middleware_chain = handler