天天看點
傳回

詳解ASP.NET Core實作的檔案斷點上傳下載下傳

作者：小乖獸技術
詳解ASP.NET Core實作的檔案斷點上傳下載下傳

場景分析

檔案的斷點續傳功能可以應用于需要上傳或下載下傳大檔案的場景。在網絡狀況不佳或者檔案較大時，一次性上傳或下載下傳整個檔案可能會耗費大量時間和帶寬，并且可能會導緻因中斷而失敗的情況發生。通過實作檔案的斷點續傳功能，可以将大檔案分割成小塊，分别上傳或下載下傳，即使在網絡出現問題時也可以通過上傳或下載下傳已經完成的檔案塊來繼續未完成的操作，減少耗時和帶寬消耗，提高使用者體驗。常見的應用場景如：視訊、音頻檔案的上傳或下載下傳，大型軟體的更新或下載下傳，雲盤等檔案存儲服務等等。

詳解ASP.NET Core實作的檔案斷點上傳下載下傳

設計思路

在asp.net core 中實作檔案的斷點續傳功能，需要進行以下步驟：

  1. 用戶端與服務端建立連接配接；
  2. 用戶端上傳檔案時，将檔案切割成小塊并分别傳輸到服務端；
  3. 服務端接收到檔案塊後進行校驗和整合；
  4. 用戶端下載下傳檔案時，請求服務端指定位置開始的檔案塊，并按照順序拼接成完整檔案。

具體實作思路如下：

斷點上傳

  1. 用戶端上傳檔案時，服務端接收到請求，在服務端建立對應的檔案，并記錄上傳時間、檔案大小等資訊；
  2. 将接收到的檔案塊儲存到實體檔案中，每次上傳塊的時候都應該壓縮這個塊，以減少帶寬的消耗；
  3. 服務端傳回給用戶端成功接收的檔案塊編号和檔案處理情況。

斷點下載下傳

  1. 用戶端下載下傳檔案時，服務端接收到請求，讀取檔案對應的塊資訊，判斷用戶端需要下載下傳的檔案塊範圍是否存在于服務端的檔案塊資訊中；
  2. 如果該檔案塊不存在，則傳回錯誤資訊；
  3. 如果該檔案塊存在，則讀取對應檔案塊并傳回給用戶端。

實作流程

斷點上傳

建立上傳控制器

在 ASP.NET Core 應用程式中，建立一個名為 UploadController 的控制器。在該控制器中，使用 ApiControllerAttribute 特性聲明該控制器為 Web API 控制器。

[ApiController]
[Route("[controller]")]
public class UploadController : ControllerBase

實作上傳方法

在控制器中實作下面的代碼。

[HttpPost]
public async Task<IActionResult> Upload(IFormFile file, int chunkIndex, int totalChunks)
{
    if (file == null || file.Length == 0)
    {
        return BadRequest("This request does not have any body");
    }

    // create the folder if it doesn't exist yet
    string folderPath = Path.Combine(Directory.GetCurrentDirectory(), "Uploads");
    if (!Directory.Exists(folderPath))
    {
        Directory.CreateDirectory(folderPath);
    }

    string filePath = Path.Combine(folderPath, file.FileName);

    using (FileStream stream = new FileStream(filePath, FileMode.Create, FileAccess.Write, FileShare.Write))
    {
        await file.CopyToAsync(stream);
    }

    return Ok();
}

實作斷點上傳邏輯

在上面的代碼中，我們隻是将整個檔案儲存到了伺服器上。現在，我們需要實作斷點上傳的邏輯。斷點上傳指的是，将大檔案分成多個小塊，并逐個上傳。

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
using System.IO;
using System.Threading.Tasks;

namespace ResumeTransfer
{
    [Route("api/[controller]")]
    [ApiController]
    public class UploadController : ControllerBase
    {
        private const string UploadsFolder = "Uploads";

        [HttpPost]
        public async Task<IActionResult> Upload(IFormFile file, int? chunkIndex, int? totalChunks)
        {
            if (file == null || file.Length == 0)
            {
                return BadRequest("This request does not have any body");
            }

            // Check if the chunk index and total chunks are provided.
            if (!chunkIndex.HasValue || chunkIndex.Value < 0 || !totalChunks.HasValue || totalChunks.Value <= 0)
            {
                return BadRequest("Invalid chunk index or total chunks");
            }

            // Create folder for upload files if not exists
            string folderPath = Path.Combine(Directory.GetCurrentDirectory(), UploadsFolder);
            if (!Directory.Exists(folderPath))
            {
                Directory.CreateDirectory(folderPath);
            }

            string filePath = Path.Combine(folderPath, file.FileName);

            // Check if chunk already exists
            if (System.IO.File.Exists(GetChunkFileName(filePath, chunkIndex.Value)))
            {
                return Ok();
            }

            using (FileStream stream = new FileStream(GetChunkFileName(filePath, chunkIndex.Value), FileMode.Create, FileAccess.Write, FileShare.Write))
            {
                await file.CopyToAsync(stream);
            }

            if (chunkIndex == totalChunks - 1)
            {
                // All the chunks have been uploaded, merge them into a single file
                MergeChunks(filePath, totalChunks.Value);
            }

            return Ok();
        }

        private void MergeChunks(string filePath, int totalChunks)
        {
            using (var finalStream = new FileStream(filePath, FileMode.CreateNew))
            {
                for (int i = 0; i < totalChunks; i++)
                {
                    var chunkFileName = GetChunkFileName(filePath, i);
                    using (var stream = new FileStream(chunkFileName, FileMode.Open))
                    {
                        stream.CopyTo(finalStream);
                    }
                    System.IO.File.Delete(chunkFileName);
                }
            }
        }

        private string GetChunkFileName(string filePath, int chunkIndex)
        {
            return #34;{filePath}.part{chunkIndex.ToString().PadLeft(5, '0')}";
        }
    }
}

用戶端上傳

用戶端使用 axios 庫進行檔案上傳。使用下面的代碼，将檔案分塊并管理每個檔案塊的大小和數量。

// get file size and name
const fileSize = file.size;
const fileName = file.name;

// calculate chunk size
const chunkSize = 10 * 1024 * 1024; // 10MB

// calculate total chunks
const totalChunks = Math.ceil(fileSize / chunkSize);

// chunk upload function
const uploadChunk = async (chunkIndex) => {
  const start = chunkIndex * chunkSize;
  const end = Math.min((chunkIndex + 1) * chunkSize, fileSize);

  const formData = new FormData();
  formData.append("file", file.slice(start, end));
  formData.append("chunkIndex", chunkIndex);
  formData.append("totalChunks", totalChunks);

  await axios.post("/upload", formData);
};

for (let i = 0; i < totalChunks; i++) {
  await uploadChunk(i);
}

斷點下載下傳

建立下載下傳控制器

建立一個名為 DownloadController 的控制器，使用 ApiControllerAttribute 特性聲明該控制器為 Web API 控制器。

[ApiController]
[Route("[controller]")]
public class DownloadController : ControllerBase

實作斷點下載下傳邏輯

在控制器中實作下面的代碼。

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System;
using System.IO;

namespace ResumeTransfer
{
    [Route("api/[controller]")]
    [ApiController]
    public class DownloadController : ControllerBase
    {
        private const string UploadsFolder = "Uploads";

        [HttpGet("{fileName}")]
        public IActionResult Download(string fileName, long? startByte = null, long? endByte = null)
        {
            if (string.IsNullOrEmpty(fileName))
            {
                return BadRequest("Invalid file name");
            }

            string filePath = Path.Combine(Directory.GetCurrentDirectory(), UploadsFolder, fileName);

            if (!System.IO.File.Exists(filePath))
            {
                return NotFound();
            }

            long contentLength = new System.IO.FileInfo(filePath).Length;

            // Calculate the range to download.
            if (startByte == null)
            {
                startByte = 0;
            }
            if (endByte == null)
            {
                endByte = contentLength - 1;
            }

            // Adjust the startByte and endByte to be within the range of the file size.
            if (startByte.Value < 0 || startByte.Value >= contentLength || endByte.Value < startByte.Value || endByte.Value >= contentLength)
            {
                Response.Headers.Add("Content-Range", #34;bytes */{contentLength}");
                return new StatusCodeResult(416); // Requested range not satisfiable
            }

            // Set the Content-Disposition header to enable users to save the file.
            Response.Headers.Add("Content-Disposition", #34;inline; filename={fileName}");

            Response.StatusCode = 206; //Partial Content

            Response.Headers.Add("Accept-Ranges", "bytes");

            long length = endByte.Value - startByte.Value + 1;
            Response.Headers.Add("Content-Length", length.ToString());

            // Send the file data in a range of bytes, if requested
            byte[] buffer = new byte[1024 * 1024];
            using (FileStream stream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.Read))
            {
                stream.Seek(startByte.Value, SeekOrigin.Begin);
                int bytesRead;
                while (length > 0 && (bytesRead = stream.Read(buffer, 0, (int)Math.Min(buffer.Length, length))) > 0)
                {
                    // Check if the client has disconnected.
                    if (!Response.HttpContext.Response.Body.CanWrite)
                    {
                        return Ok();
                    }

                    Response.Body.WriteAsync(buffer, 0, bytesRead);
                    length -= bytesRead;
                }
            }

            return new EmptyResult();
        }
    }
}

用戶端下載下傳

用戶端使用 axios 庫進行檔案下載下傳。使用下面的代碼，将要下載下傳的檔案拆分成小塊，并按照順序下載下傳。

const CHUNK_SIZE = 1024 * 1024 * 5; // 5MB

const downloadChunk = async (chunkIndex) => {
  const res = await axios.get(`/download?fileName=${fileName}&chunkIndex=${chunkIndex}`, {
    responseType: "arraybuffer",
  });
  const arrayBuffer = res.data;

  const start = chunkIndex * CHUNK_SIZE;
  const end = start + CHUNK_SIZE;
  const blob = new Blob([arrayBuffer]);

  const url = URL.createObjectURL(blob);

  const a = document.createElement("a");
  a.style.display = "none";
  a.href = url;
  a.download = fileName;
  document.body.appendChild(a);
  a.click();
  URL.revokeObjectURL(url);
};

for (let i = 0; i < totalChunks; i++) {
  await downloadChunk(i);
}

單元測試

在我們的代碼中實作單元測試可以確定代碼的正确性，并且可以減少手工測試的負擔。我們可以使用 Microsoft.VisualStudio.TestTools.UnitTesting（在 .NET Core 中，也可以使用 xUnit 或 NUnit 進行單元測試）進行單元測試。

下面是一個簡單的上傳控制器單元測試示例：

using ResumeTransfer;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Threading.Tasks;
using Xunit;
using Microsoft.Extensions.Primitives;
using Moq;

namespace ResumeTransfer.Tests
{
    public class UploadControllerTests
    {
        [Fact]
        public async Task Upload_ReturnsBadRequest_WhenNoFileIsSelected()
        {
            // Arrange
            var formCollection = new FormCollection(new Dictionary<string, StringValues>(), new FormFileCollection());
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection);
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object } };

            // Act
            var result = await controller.Upload(null, 0, 1);

            // Assert
            Assert.IsType<BadRequestObjectResult>(result);
        }

        [Fact]
        public async Task Upload_ReturnsBadRequest_WhenInvalidChunkIndexOrTotalChunksIsProvided()
        {
            // Arrange
            var fileName = "test.txt";
            var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
            var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
            var formCollection = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "0" }
            }, new FormFileCollection { formFile });
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection);
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object } };

            // Act
            var result = await controller.Upload(formFile, 0, 0);

            // Assert
            Assert.IsType<BadRequestObjectResult>(result);
        }

        [Fact]
        public async Task Upload_UploadsChunk_WhenChunkDoesNotExist()
        {
            // Arrange
            var fileName = "test.txt";
            var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
            var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
            var formCollection = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "1" }
            }, new FormFileCollection { formFile });
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection);
            var uploadsFolder = Path.Combine(Directory.GetCurrentDirectory(), "uploads");
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object }, UploadsFolder = uploadsFolder };

            // Act
            var result = await controller.Upload(formFile, 0, 1);
            var uploadedFilePath = Path.Combine(uploadsFolder, fileName);

            // Assert
            Assert.IsType<OkResult>(result);
            Assert.True(File.Exists(uploadedFilePath));

            using (var streamReader = new StreamReader(File.OpenRead(uploadedFilePath)))
            {
                var content = await streamReader.ReadToEndAsync();
                Assert.Equal("This is a test file.", content);
            }

            File.Delete(uploadedFilePath);
        }

        [Fact]
        public async Task Upload_UploadsChunk_WhenChunkExists()
        {
            // Arrange
            var fileName = "test.txt";
            var fileStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file."));
            var formFile = new FormFile(fileStream, 0, fileStream.Length, "Data", fileName);
            var formCollection1 = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "2" }
            }, new FormFileCollection { formFile });
            var context = new Mock<HttpContext>();
            context.SetupGet(x => x.Request.Form).Returns(formCollection1);
            var uploadsFolder = Path.Combine(Directory.GetCurrentDirectory(), "uploads");
            var controller = new UploadController { ControllerContext = new ControllerContext { HttpContext = context.Object }, UploadsFolder = uploadsFolder };

            // Act
            var result1 = await controller.Upload(formFile, 0, 2);

            // Assert
            Assert.IsType<OkResult>(result1);

            // Arrange
            var formCollection2 = new FormCollection(new Dictionary<string, StringValues>
            {
                { "chunkIndex", "1" },
                { "totalChunks", "2" }
            }, new FormFileCollection { formFile });
            context.SetupGet(x => x.Request.Form).Returns(formCollection2);

            // Act
            var result2 = await controller.Upload(formFile, 1, 2);
            var uploadedFilePath = Path.Combine(uploadsFolder, fileName);

            // Assert
            Assert.IsType<OkResult>(result2);
            Assert.True(File.Exists(uploadedFilePath));

            using (var streamReader = new StreamReader(File.OpenRead(uploadedFilePath)))
            {
                var content = await streamReader.ReadToEndAsync();
                Assert.Equal("This is a test file.This is a test file.", content);
            }

            File.Delete(uploadedFilePath);
        }
    }
}
詳解ASP.NET Core實作的檔案斷點上傳下載下傳

性能測試

為了實作更好的性能和響應時間，我們可以使用 BenchmarkDotNet 進行性能測試，以便找到性能瓶頸并對代碼進行優化。

下面是一個簡單的上傳控制器性能測試示例：

using BenchmarkDotNet.Attributes;
using ResumeTransfer;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http.Internal;

namespace ResumeTransfer.Benchmarks
{
    [MemoryDiagnoser]
    public class UploadControllerBenchmarks
    {
        private readonly UploadController _controller;
        private readonly IFormFile _testFile;

        public UploadControllerBenchmarks()
        {
            _controller = new UploadController();
            _testFile = new FormFile(new MemoryStream(System.Text.Encoding.UTF8.GetBytes("This is a test file")), 0, 0, "TestFile", "test.txt");
        }

        [Benchmark]
        public async Task<IActionResult> UploadSingleChunk()
        {
            var formCollection = new FormCollection(new System.Collections.Generic.Dictionary<string, Microsoft.Extensions.Primitives.StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", "1" }
            }, new FormFileCollection { _testFile });
            var request = new DefaultHttpContext();
            request.Request.Form = formCollection;
            _controller.ControllerContext = new ControllerContext { HttpContext = request };

            return await _controller.Upload(_testFile, 0, 1);
        }

        [Benchmark]
        public async Task<IActionResult> UploadMultipleChunks()
        {
            var chunkSizeBytes = 10485760; // 10 MB
            var totalFileSizeBytes = 52428800; // 50 MB
            var totalChunks = (int)Math.Ceiling((double)totalFileSizeBytes / chunkSizeBytes);

            for (var i = 0; i < totalChunks; i++)
            {
                var chunkStartByte = i * chunkSizeBytes;
                var chunkEndByte = Math.Min(chunkStartByte + chunkSizeBytes - 1, totalFileSizeBytes - 1);

                var chunkFileContent = new byte[chunkEndByte - chunkStartByte + 1];
                using (var memoryStream = new MemoryStream(chunkFileContent))
                {
                    using (var binaryWriter = new BinaryWriter(memoryStream))
                    {
                        binaryWriter.Write(chunkStartByte);
                        binaryWriter.Write(chunkEndByte);
                    }
                }

                var chunkFileName = #34;{_testFile.FileName}.part{i.ToString().PadLeft(5, '0')}";
                var chunkFilePath = Path.Combine(Directory.GetCurrentDirectory(), ChunkUploadController.UploadsFolder, chunkFileName);

                using (var fileStream = new FileStream(chunkFilePath, FileMode.Create, FileAccess.Write, FileShare.Write))
                {
                    await fileStream.WriteAsync(chunkFileContent);
                    await fileStream.FlushAsync();
                }
            }

            var formCollection = new FormCollection(new System.Collections.Generic.Dictionary<string, Microsoft.Extensions.Primitives.StringValues>
            {
                { "chunkIndex", "0" },
                { "totalChunks", totalChunks.ToString() }
            }, new FormFileCollection { _testFile });
            var request = new DefaultHttpContext();
            request.Request.Form = formCollection;
            _controller.ControllerContext = new ControllerContext { HttpContext = request };

            return await _controller.Upload(_testFile, 0, totalChunks);
        }
    }
}

運作上面的代碼後，将會輸出詳細的性能測試結果。

詳解ASP.NET Core實作的檔案斷點上傳下載下傳

根據測試結果，我們可以找到性能瓶頸，并對代碼進行優化，以達到更高的性能和更快的響應時間。

ASP.NET Core 斷點上傳 上傳下載下傳 P.NET Core .net core Core .net
上一篇: 币安官方解讀什麼是币安鍊二層網絡opBNB
下一篇: Excel怎麼快速删除工作表中的空白行？

繼續閱讀