This thing is like this, although iN itself doesn't get NAS or anything at home, but there is still a need to watch videos at home. It's also lazy, too lazy to search for download resources in various forums. So I made a video site at home to collect video feeds through the function of source scraping.
Many, many people can do this, and there's no need to write any tutorials.
However, sometimes some video sources are not smooth due to network blocking, which is not the case in recent days, when "Celebrating More Than Years 2" was collected, due to the popularity of this drama, the source URL went down, and the server thrown on the VPS could not search for any resources......
This actually stems from the limitations of various service API ports.
How?
We can use cloudflare's Workers to handle this. The way to do this is to simply add a shell to our URL request.
Let's talk about the principle:
Most URL-based APIs don't usually care much about the origin of the request, but the source of frequent requests is often blocked. This causes us to get an error when calling through the web client.
SIMILARLY, MOST WEB API-BASED CONTENT CAN BE MODIFIED, AND IF MODIFIED, IT OFTEN INTRODUCES INSECURITY ISSUES.
For this phenomenon, we can use network tools to add a "shell" to the target of the request, to be precise, URL forwarding - send the URL address we want to request to a Worker script made by ourselves, and then let the Worker script request the real service, and then return the result of the request to our own server through the Worker.
At this point, you can have a vague framework for the concept of Worker - Cloudflare Workers is an edge computing technology that enables developers to run JavaScript code on Cloudflare's global network. Unlike traditional server hosting, Cloudflare Workers allows code to be executed in the location closest to the user, improving responsiveness and performance.
What we call "adding a shell" is actually a kind of edge computing in the network, and the data processing and computing resources can be distributed in the edge nodes of the network using Woker, rather than being concentrated in the NAS at home or the VPS in the data center.
By adding an intermediate processing layer, we can make another layer of judgment between the source data and the service to improve security. At the same time, most workers do not have fixed IP addresses, and access to Web APIs can also avoid the problem of blocking IPs due to too frequent access.
To do this, do the following:
Precondition:
1. You have a cloudflare account to set up a Worker
2. YOU HAVE THE WEB API ADDRESS YOU NEED TO ACCESS.
Non-sufficient requirements: a little knowledge of JS scripting.
Method:
Go to the cloudflare console and create a new worker in the Worker settings
Name the worker:
This name is an easy-to-remember, tag name that you can quickly recognize, and it is also identified by this name when accessing the Worker.
At the same time, in the code preview, cloudflare will generate a simple hello world script, so that our deployed worker can at least return some information.
There is no need to make any other changes in this interface, just click the Deploy button and we can deploy a Worker script.
After the first deployment is successful, we can use the edit code button to edit the code of the newly created worker
Going into the code editor, Cloudflare provides a simple code writing environment that is divided into three parts
1: Write code
2. Request to send
3. Results
For example, the video capture worker that iN does today has only a few lines of code:
The principle is very simple, read the URL parameters requested to the worker, replace them with targetUrl, and then use the fetch (targetUrl) method to obtain the information that the original WEB API should return. At this point, we can get the reply from the original Web API and pass the reply to our own server or NAS as it is.
Replace the URL part of the parameters required by the original API with the address of our own Worker, and keep the original parameters unchanged, and you can work.
原始API URL的形式： AAAAA/参数
Worker的主机：CCCCC
Changed form: CCCCC/parameter
It's as simple as that.
Say the benefits:
The first is to hide the address of our own host, after all, many organizations or individuals who currently provide APIs are still not very trustworthy, and there is no need to let the other party know our own IP
Secondly, the speed is fast, because the Worker is built in the CDN network and has its own mirrors in various locations around the world, the network distance from the Worker to the target is much smaller than the distance from our own host to the target. This guarantees the speed of access.
Thirdly, the content of the target API is auditable. We can use the program to audit the returned content before it reaches our own system, and then add it to our own system. Such security could be improved.
Finally, let's talk about the cost: 100,000 visits per day are free, and it is difficult for general household and personal use to send out 100,000 visits in a day. At present, the Worker who has been using it for more than half a year has not paid for this function!