前言
前情回顧:上一篇我們遺留了兩個問題,一個是未完全實作斷點續傳,另外則是在響應時是傳回StreamContent還是PushStreamContent呢?這一節我們重點來解決這兩個問題,同時就在此過程中需要注意的地方一并指出,若有錯誤之處,請指出。
StreamContent compare to PushStreamContent
我們來看看StreamContent代碼,如下:
public class StreamContent : HttpContent { // Fields private int bufferSize; private Stream content; private bool contentConsumed; private const int defaultBufferSize = 0x1000; private long start; // Methods public StreamContent(Stream content); ] public StreamContent(Stream content, int bufferSize); protected override Task<Stream> CreateContentReadStreamAsync(); protected override void Dispose(bool disposing); private void PrepareContent(); protected override Task SerializeToStreamAsync(Stream stream, TransportContext context); protected internal override bool TryComputeLength(out long length); // Nested Types private class ReadOnlyStream : DelegatingStream {......} }
似乎沒有什麼可看的,但是有一句話我們需要注意,如下:
private const int defaultBufferSize = 0x1000;
在StreamContent的第二個構造函數為
public StreamContent(Stream content, int bufferSize);
上述給定的預設一次性輸入到緩沖區大小為4k,這對我們有何意義呢?當我們寫入到響應中時,一般我們直接利用的是第一個構造函數,如下:
var response = new HttpResponseMessage(); response.Content = new StreamContent(fileStream);
到這裡我們明白了這麼做是有問題的,當下載下傳時預設讀取的是4k,如果檔案比較大下載下傳的時間則有延長,是以我們在傳回時一定要給定緩沖大小,那麼給定多少呢?為達到更好的性能最多是80k,如下:
private const int BufferSize = 80 * 1024; response.Content = new StreamContent(fileStream, BufferSize);
此時下載下傳的速度則有很大的改善,有人就說了為何是80k呢?這個問題我也不知道,老外驗證過的,這是連結【.NET Asynchronous stream read/write】。
好了說完StreamContent,接下來我們來看看PushStreamContent,從字面意思來為推送流内容,難道是充分利用了緩沖區嗎,猜測可以有,就怕沒有任何想法,我們用源碼來證明看看。
我們隻需看看WebHost模式下對于緩沖政策是怎麼選擇的,我們看看此類 WebHostBufferPolicySelector 實作,代碼如下:
/// <summary> /// Provides an implementation of <see cref="IHostBufferPolicySelector"/> suited for use /// in an ASP.NET environment which provides direct support for input and output buffering. /// </summary> public class WebHostBufferPolicySelector : IHostBufferPolicySelector { ....../// <summary> /// Determines whether the host should buffer the <see cref="HttpResponseMessage"/> entity body. /// </summary> /// <param name="response">The <see cref="HttpResponseMessage"/>response for which to determine /// whether host output buffering should be used for the response entity body.</param> /// <returns><c>true</c> if buffering should be used; otherwise a streamed response should be used.</returns> public virtual bool UseBufferedOutputStream(HttpResponseMessage response) { if (response == null) { throw Error.ArgumentNull("response"); } // Any HttpContent that knows its length is presumably already buffered internally. HttpContent content = response.Content; if (content != null) { long? contentLength = content.Headers.ContentLength; if (contentLength.HasValue && contentLength.Value >= 0) { return false; } // Content length is null or -1 (meaning not known). // Buffer any HttpContent except StreamContent and PushStreamContent return !(content is StreamContent || content is PushStreamContent); } return false; } }
從上述如下一句可以很明顯的知道:
return !(content is StreamContent || content is PushStreamContent);
除了StreamContent和PushStreamContent的HttpContent之外,其餘都進行緩沖,是以二者的差別不在于緩沖,那到底是什麼呢?好了我們還未檢視PushStreamContent的源碼,我們繼續往下走,檢視其源代碼如下,我們僅僅隻看關于這個類的描述以及第一個構造函數即可,如下:
/// <summary> /// Provides an <see cref="HttpContent"/> implementation that exposes an output <see cref="Stream"/> /// which can be written to directly. The ability to push data to the output stream differs from the /// <see cref="StreamContent"/> where data is pulled and not pushed. /// </summary> public class PushStreamContent : HttpContent { private readonly Func<Stream, HttpContent, TransportContext, Task> _onStreamAvailable; /// <summary> /// Initializes a new instance of the <see cref="PushStreamContent"/> class. The /// <paramref name="onStreamAvailable"/> action is called when an output stream /// has become available allowing the action to write to it directly. When the /// stream is closed, it will signal to the content that is has completed and the /// HTTP request or response will be completed. /// </summary> /// <param name="onStreamAvailable">The action to call when an output stream is available.</param> public PushStreamContent(Action<Stream, HttpContent, TransportContext> onStreamAvailable) : this(Taskify(onStreamAvailable), (MediaTypeHeaderValue)null) { } ...... }
對于此類的描述大意是:PushStreamContent與StreamContent的不同在于,PushStreamContent在于将資料push【推送】到輸出流中,而StreamContent則是将資料從流中【拉取】。
貌似有點晦澀,我們來舉個例子,在webapi中我們常常這樣做,讀取檔案流并傳回到響應流中,若是StreamContent,我們會如下這樣做:
response.Content = new StreamContent(File.OpenRead(filePath));
上面的釋義我用大括号着重括起,StreamContent着重于【拉取】,當響應時此時将從檔案流寫到輸出流,通俗一點說則是我們需要從檔案流中去擷取資料并寫入到輸出流中。我們再來看看PushStreamContent的用法,如下:
XDocument xDoc = XDocument.Load("cnblogs_backup.xml", LoadOptions.None); PushStreamContent xDocContent = new PushStreamContent( (stream, content, context) => { xDoc.Save(stream); stream.Close(); }, "application/xml");
PushStreamContent着重于【推送】,當我們加載xml檔案時,當我們一旦進行儲存時此時則會将資料推送到輸出流中。
二者差別在于:StreamContent從流中【拉取】資料,而PushStreamContent則是将資料【推送】到流中。
那麼此二者應用的場景是什麼呢?
(1)對于下載下傳檔案我們則可以通過StreamContent來實作直接從流中拉取,若下載下傳視訊流此時則應該利用PushStreamContent來實作,因為未知伺服器視訊資源的長度,此視訊資源來源于别的地方。
(2)資料量巨大,發送請求到webapi時利用PushStreamContent。
當發送請求時,常常序列化資料并請求webapi,我們可能這樣做:
var client = new HttpClient(); string json = JsonConvert.SerializeObject(data); var response = await client.PostAsync(uri, new StringContent(json));
當資料量比較小時沒問題,若資料比較大時進行序列化此時則将序列化的字元串加載到記憶體中,鑒于此這麼做不可行,此時我們應該利用PushStreamContent來實作。
var client = new HttpClient(); var content = new PushStreamContent((stream, httpContent, transportContext) => { var serializer = new JsonSerializer(); using (var writer = new StreamWriter(stream)) { serializer.Serialize(writer, data); } }); var response = await client.PostAsync(uri, content);
為什麼要這樣做呢?我們再來看看源碼,裡面存在這樣一個方法。
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context);
其内部實作利用異步狀态機實作,是以當資料量巨大時利用PushStreamContent來傳回将會有很大的改善,至此,關于二者的差別以及常見的應用場景已經叙述完畢,接下來我們繼續斷點續傳問題。
斷點續傳改進
上一篇我們講過擷取Range屬性中的集合通過如下:
request.Headers.Range
我們隻取該集合中的第一個範圍元素,通過如下
RangeItemHeaderValue range = rangeHeader.Ranges.First();
此時我們忽略了傳回的該範圍對象中有目前下載下傳的進度
range.From.HasValue range.To.HasValue
我們擷取二者的值然後進行重寫Stream實時讀取剩餘部分,下面我們一步一步來看。
定義檔案操作接口
public interface IFileProvider { bool Exists(string name); FileStream Open(string name); long GetLength(string name); }
實作該操作檔案接口
public class FileProvider : IFileProvider { private readonly string _filesDirectory; private const string AppSettingsKey = "DownloadDir"; public FileProvider() { var fileLocation = ConfigurationManager.AppSettings[AppSettingsKey]; if (!String.IsNullOrWhiteSpace(fileLocation)) { _filesDirectory = fileLocation; } } /// <summary> /// 判斷檔案是否存在 /// </summary> /// <param name="name"></param> /// <returns></returns> public bool Exists(string name) { string file = Directory.GetFiles(_filesDirectory, name, SearchOption.TopDirectoryOnly) .FirstOrDefault(); return true; } /// <summary> /// 打開檔案 /// </summary> /// <param name="name"></param> /// <returns></returns> public FileStream Open(string name) { var fullFilePath = Path.Combine(_filesDirectory, name); return File.Open(fullFilePath, FileMode.Open, FileAccess.Read, FileShare.Read); } /// <summary> /// 擷取檔案長度 /// </summary> /// <param name="name"></param> /// <returns></returns> public long GetLength(string name) { var fullFilePath = Path.Combine(_filesDirectory, name); return new FileInfo(fullFilePath).Length; } }
擷取範圍對象中的值進行指派給封裝的對象
public class FileInfo { public long From; public long To; public bool IsPartial; public long Length; }
下載下傳控制器,對檔案操作進行初始化
public class FileDownloadController : ApiController { private const int BufferSize = 80 * 1024; private const string MimeType = "application/octet-stream"; public IFileProvider FileProvider { get; set; } public FileDownloadController() { FileProvider = new FileProvider(); } ...... }
接下來則是檔案下載下傳的邏輯,首先判斷請求檔案是否存在,然後擷取檔案的長度
if (!FileProvider.Exists(fileName)) { throw new HttpResponseException(HttpStatusCode.NotFound); } long fileLength = FileProvider.GetLength(fileName);
将請求中的範圍對象From和To的值并判斷目前已經下載下傳進度以及剩餘進度
private FileInfo GetFileInfoFromRequest(HttpRequestMessage request, long entityLength) { var fileInfo = new FileInfo { From = 0, To = entityLength - 1, IsPartial = false, Length = entityLength }; var rangeHeader = request.Headers.Range; if (rangeHeader != null && rangeHeader.Ranges.Count != 0) { if (rangeHeader.Ranges.Count > 1) { throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable); } RangeItemHeaderValue range = rangeHeader.Ranges.First(); if (range.From.HasValue && range.From < 0 || range.To.HasValue && range.To > entityLength - 1) { throw new HttpResponseException(HttpStatusCode.RequestedRangeNotSatisfiable); } fileInfo.From = range.From ?? 0; fileInfo.To = range.To ?? entityLength - 1; fileInfo.IsPartial = true; fileInfo.Length = entityLength; if (range.From.HasValue && range.To.HasValue) { fileInfo.Length = range.To.Value - range.From.Value + 1; } else if (range.From.HasValue) { fileInfo.Length = entityLength - range.From.Value + 1; } else if (range.To.HasValue) { fileInfo.Length = range.To.Value + 1; } } return fileInfo; }
在響應頭資訊中的對象ContentRangeHeaderValue設定目前下載下傳進度以及其他響應資訊
private void SetResponseHeaders(HttpResponseMessage response, FileInfo fileInfo, long fileLength, string fileName) { response.Headers.AcceptRanges.Add("bytes"); response.StatusCode = fileInfo.IsPartial ? HttpStatusCode.PartialContent : HttpStatusCode.OK; response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment"); response.Content.Headers.ContentDisposition.FileName = fileName; response.Content.Headers.ContentType = new MediaTypeHeaderValue(MimeType); response.Content.Headers.ContentLength = fileInfo.Length; if (fileInfo.IsPartial) { response.Content.Headers.ContentRange = new ContentRangeHeaderValue(fileInfo.From, fileInfo.To, fileLength); } }
最重要的一步則是将FileInfo對象的值傳遞給我們自定義實作的流監控目前下載下傳進度。
public class PartialContentFileStream : Stream { private readonly long _start; private readonly long _end; private long _position; private FileStream _fileStream; public PartialContentFileStream(FileStream fileStream, long start, long end) { _start = start; _position = start; _end = end; _fileStream = fileStream; if (start > 0) { _fileStream.Seek(start, SeekOrigin.Begin); } } /// <summary> /// 将緩沖區資料寫到檔案 /// </summary> public override void Flush() { _fileStream.Flush(); } /// <summary> /// 設定目前下載下傳位置 /// </summary> /// <param name="offset"></param> /// <param name="origin"></param> /// <returns></returns> public override long Seek(long offset, SeekOrigin origin) { if (origin == SeekOrigin.Begin) { _position = _start + offset; return _fileStream.Seek(_start + offset, origin); } else if (origin == SeekOrigin.Current) { _position += offset; return _fileStream.Seek(_position + offset, origin); } else { throw new NotImplementedException("SeekOrigin.End未實作"); } } /// <summary> /// 依據偏離位置讀取 /// </summary> /// <param name="buffer"></param> /// <param name="offset"></param> /// <param name="count"></param> /// <returns></returns> public override int Read(byte[] buffer, int offset, int count) { int byteCountToRead = count; if (_position + count > _end) { byteCountToRead = (int)(_end - _position) + 1; } var result = _fileStream.Read(buffer, offset, byteCountToRead); _position += byteCountToRead; return result; } public override IAsyncResult BeginRead(byte[] buffer, int offset, int count, AsyncCallback callback, object state) { int byteCountToRead = count; if (_position + count > _end) { byteCountToRead = (int)(_end - _position); } var result = _fileStream.BeginRead(buffer, offset, count, (s) => { _position += byteCountToRead; callback(s); }, state); return result; } ...... }
更新上述下載下傳的完整邏輯
public HttpResponseMessage GetFile(string fileName) { fileName = "HBuilder.windows.5.2.6.zip"; if (!FileProvider.Exists(fileName)) { throw new HttpResponseException(HttpStatusCode.NotFound); } long fileLength = FileProvider.GetLength(fileName); var fileInfo = GetFileInfoFromRequest(this.Request, fileLength); var stream = new PartialContentFileStream(FileProvider.Open(fileName), fileInfo.From, fileInfo.To); var response = new HttpResponseMessage(); response.Content = new StreamContent(stream, BufferSize); SetResponseHeaders(response, fileInfo, fileLength, fileName); return response; }
下面我們來看看示範結果:

好了,到了這裡我們也得到了我們想要的結果。
總結
本節我們将上節遺留的問題一一進行比較詳細的叙述并最終解決,是不是就這麼完全結束了呢?那本節定義為中篇豈不是不對頭了,本節是在web端進行下載下傳,下節我們利用webclient來進行斷點續傳。想了想無論是mvc上傳下載下傳,還是利用webapi來上傳下載下傳又或者是将mvc和webapi結合來上傳下載下傳基本都已經囊括,這都算是在項目中比較常用的吧,是以也就花了很多時間去研究。對于webapi的斷點續傳關鍵它本身就提供了比較多的api來給我們調用,是以還是很不錯,webapi一個很輕量的服務架構,你值得擁有see u,反正周末,喲,不早了,休息休息。
所有的選擇不過是為了下一次選擇做準備