在上面提到了django内部的数据流转,那么就顺便说说request在django内部流转,最终生成response。下面这幅图就很好说明了数据处理过程。
1. 用户在浏览器输入url时,浏览器会生成请求报文通过 OSI七层网络模型 发给服务
2. 请求到达服务器时,先通过 wsgi协议 将http请求数据转化为python请求对象
3. 请求到达request 中间件,中间件对request请求做预处理检查,如果检查不通过则直接返回 response,否则进入下一步
4. 到达django路由系统,找到对应的视图函数
5. 视图函数通过ORM访问底层数据,并进行相应处理返回response
OSI七层网络模型
当用户在浏览器输入网址时,浏览器会将数据封装成请求数据包括请求头和请求体(应用层),经过转换、压缩、加密,请求数据被转化成 message报文(表示层),在服务器和客户端建立会话连接(会话层)。
(传输层)解封message报文后得到segment数据段,根据tcp头或udp头(源端口和目的端口,传输数据序号)进行传输。(网络层)解封上层数据后得到packet网络包,根据目的ip和源ip进行路由转发。在每段路由转发过程,(数据链路层)数据包继续被解封成frame数据帧,并根据以太网数据进行节点对节点之间的传输。此时,(物理层)数据将会以bit比特流方式传输。
到达目的地址后,数据将会像上述逆序封装。最终服务器收到request请求数据
小知识点:
非官方语言,仅小故事记忆
1. 三步握手建立连接
某天,A想打电话给B,但又怕影响到B工作。于是,
- A发了条SYN短信给B:我想打电话给你,可以吗
- B回复ACK信息:刚才的信息已经收到了,并约定SYN通信
- A回复ACK信息:收到,到时间联系。
2. 四步挥手断开连接
两人聊了一段时间,A想挂断电话。
- A发送FIN:我没有要说的了,挂断吧
- B发送ACK:我知道了,我还有几句要说的
- B巴拉巴拉说了一通后,回复FIN、ACK:好了,我们结束通话吧
- A回复ACK:好,我挂断了
wsgi协议
WSGI协议(web server gateway interface)是一种通信协议,主要包括两部分:
- server: 从客户端接收请求,将request转发给application,将application返回的response返回给客户端
- application: 接收由server转发的request,并将处理结果返回给server
uWSGI和gunicorn1是实现了wsgi server协议的服务器,django,flask是实现了wsgi application协议的web框架, 可以根据项目实际情况搭配使用。
那么,我就以gunicorn+django为例说明:
当我们创建django项目时,
python manage.py startproject HelloWorld
可以查看项目目录
$ tree
| -- HelloWorld
| | -- __init__.py
| | -- settings.py
| | -- urls.py
| | -- wsgi.py
| -- manage.py
启动django项目时,执行命令
gunicorn HelloWorld.wsgi
此时,gunicorn 建立起 WSGI 服务器,并与HelloWorld.wsgi application建立链接。
application 源码解析
class WSGIHandler(base.BaseHandler):
request_class = WSGIRequest
def __init__(self, *args, **kwargs):
super(WSGIHandler, self).__init__(*args, **kwargs)
# 加载中间件
self.load_middleware()
def __call__(self, environ, start_response):
# 设置环境变量信息,如DJANGO_SETTINGS_MODULE
set_script_prefix(get_script_name(environ))
# 请求处理前发送信息
signals.request_started.send(sender=self.__class__, environ=environ)
# 初始化request 对象
request = self.request_class(environ)
# 处理request,生成response(通过中间件处理)
response = self.get_response(request)
response._handler_class = self.__class__
# response数据处理
status = '%d %s' % (response.status_code, response.reason_phrase)
response_headers = [(str(k), str(v)) for k, v in response.items()]
for c in response.cookies.values():
response_headers.append((str('Set-Cookie'), str(c.output(header=''))))
# server提供的回调方法,将响应的header和status返回给server
start_response(force_str(status), response_headers)
if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'):
response = environ['wsgi.file_wrapper'](response.file_to_stream)
return response
- 虽然django本身也提供了runserver来启动wsgi server,但这种启动方式是单进程的。我更推荐使用了uwsgi\gunicorn\Apache等方式启动服务,实现用户请求数据处理和应用业务逻辑处理隔离开。同时,uwsgi\gunicorn\Apache还有高并发,高性能,多进程处理的优势。
中间件数据处理
中间件是介于request处理和response返回之间的一道处理过程,将请求数据处理和业务逻辑分离。
源码分析
在加载中间件load_middleware时,我们可以看到下述代码:
if hasattr(mw_instance, 'process_request'):
self._request_middleware.append(mw_instance.process_request)
if hasattr(mw_instance, 'process_view'):
self._view_middleware.append(mw_instance.process_view)
if hasattr(mw_instance, 'process_template_response'):
self._template_response_middleware.insert(0, mw_instance.process_template_response)
if hasattr(mw_instance, 'process_response'):
self._response_middleware.insert(0, mw_instance.process_response)
if hasattr(mw_instance, 'process_exception'):
self._exception_middleware.insert(0, mw_instance.process_exception)
这意味着中间件可以预定义5个函数: process_request 请求预处理、process_view 视图预处理、process_template_response 模板渲染处理、process_response 响应处理、 process_exception 异常处理
在这里要注意: 前两个请求函数顺序处理,后三个响应函数逆序处理。
def get_response(self, request):
urlconf = settings.ROOT_URLCONF
urlresolvers.set_urlconf(urlconf)
resolver = urlresolvers.get_resolver(urlconf)
response_is_rendered = False
try:
response = None
# 执行process_request函数,若返回response直接中断后续中间件的处理
for middleware_method in self._request_middleware:
response = middleware_method(request)
if response:
break
if response is None:
if hasattr(request, 'urlconf'):
urlconf = request.urlconf
urlresolvers.set_urlconf(urlconf)
resolver = urlresolvers.get_resolver(urlconf)
# 处理url请求
resolver_match = resolver.resolve(request.path_info)
# callback: 处理request的python函数 callback_args: 位置参数 callback_kwargs: 关键字参数字典
callback, callback_args, callback_kwargs = resolver_match
request.resolver_match = resolver_match
# 执行 process_view 函数,同样返回response,则中断后续中间件处理
for middleware_method in self._view_middleware:
response = middleware_method(request, callback, callback_args, callback_kwargs)
if response:
break
# 进入 callback 函数,执行业务逻辑;执行报错进入process_exception函数处理
if response is None:
wrapped_callback = self.make_view_atomic(callback)
try:
response = wrapped_callback(request, *callback_args, **callback_kwargs)
except Exception as e:
response = self.process_exception_by_middleware(e, request)
if response is None:
if isinstance(callback, types.FunctionType): # FBV
view_name = callback.__name__
else: # CBV
view_name = callback.__class__.__name__ + '.__call__'
raise ValueError("The view %s.%s didn't return an HttpResponse object. It returned None instead."
% (callback.__module__, view_name))
# 当response对象有 render 方法时,才会调用
if hasattr(response, 'render') and callable(response.render):
for middleware_method in self._template_response_middleware:
response = middleware_method(request, response)
# Complain if the template response middleware returned None (a common error).
if response is None:
raise ValueError(
"%s.process_template_response didn't return an "
"HttpResponse object. It returned None instead."
% (middleware_method.__self__.__class__.__name__))
# response 渲染处理
try:
response = response.render()
except Exception as e:
response = self.process_exception_by_middleware(e, request)
response_is_rendered = True
except http.Http404 as exc:
logger.warning('Not Found: %s', request.path,
extra={
'status_code': 404,
'request': request
})
if settings.DEBUG:
response = debug.technical_404_response(request, exc)
else:
response = self.get_exception_response(request, resolver, 404, exc)
except PermissionDenied as exc:
logger.warning(
'Forbidden (Permission denied): %s', request.path,
extra={
'status_code': 403,
'request': request
})
response = self.get_exception_response(request, resolver, 403, exc)
except MultiPartParserError as exc:
logger.warning(
'Bad request (Unable to parse request body): %s', request.path,
extra={
'status_code': 400,
'request': request
})
response = self.get_exception_response(request, resolver, 400, exc)
except SuspiciousOperation as exc:
# The request logger receives events for any problematic request
# The security logger receives events for all SuspiciousOperations
security_logger = logging.getLogger('django.security.%s' %
exc.__class__.__name__)
security_logger.error(
force_text(exc),
extra={
'status_code': 400,
'request': request
})
if settings.DEBUG:
return debug.technical_500_response(request, *sys.exc_info(), status_code=400)
response = self.get_exception_response(request, resolver, 400, exc)
except SystemExit:
# Allow sys.exit() to actually exit. See tickets #1023 and #4701
raise
except: # Handle everything else.
# Get the exception info now, in case another exception is thrown later.
signals.got_request_exception.send(sender=self.__class__, request=request)
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
try:
# 执行 process_response 函数
for middleware_method in self._response_middleware:
response = middleware_method(request, response)
# Complain if the response middleware returned None (a common error).
if response is None:
raise ValueError(
"%s.process_response didn't return an "
"HttpResponse object. It returned None instead."
% (middleware_method.__self__.__class__.__name__))
response = self.apply_response_fixes(request, response)
except:
signals.got_request_exception.send(sender=self.__class__, request=request)
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
response._closable_objects.append(request)
if not response_is_rendered and callable(getattr(response, 'render', None)):
response = response.render()
# 返回 response 对象
return response
总结
- gunicorn较uwsgi配置简单,易上手,且响应速度快。但uwsgi功能强大,稳定性强,吞吐量大 ↩︎