In the project, the files uploaded from the front desk (our project is just a simple restful api, so some restrictions must be done well) need to be verified, mainly in terms of size and type (extension). The current practice is to parse the file information in the form through the multiparty. I have now changed the original implementation to monitor the incoming part (related concept of multipart, similar to one of multipart/form-data) and parse the file information it contains.
However, there are still some optimization ideas. For example, if the type and size of the file can be detected before the end of the file stream, then I can reject the upload as soon as possible. However, first of all, I don’t know whether this idea is technically feasible (http supports it or not). In addition, because I’m not familiar with what curl post data shows, I can’t analyze the upload process through curl in a short period of time.
Therefore, I would like to ask these two questions.
Is this idea technically feasible?
What convenient tools can be used to analyze the sending process of http messages during file uploading (if file metadata is sent independently first, it is possible to analyze in advance).
This is not just a node-related problem, it is only because I met it when I was actually working on a node project that I asked.
The http protocol runs on tcp, so the entire formdata part of the data is always uploaded to the server sequentially.
When you get formdata data data from the back-end app, almost the entire form has been uploaded.
Unless you parse formdata’s entire io stream yourself, including http header information and the body portion of formdata.
Nodejs is not used at ordinary times, so I don’t know if there is such a module, but I know there are other similar functional modules.
There is a nginx_upload_module.
Another is lua_resty_upload, which is also a lua script running inside nginx.
There is also a tornado-stream-upload, you can have a look.
These are all processed by parsing the whole http body part as a file stream.
In this way, meta information of the file can be obtained in the front part of formdata.
Even you can put meta information in http header information. Custom header information.