I'm trying to upload files to the server in this way (below are parts of three files, a router, a controller and a script that loads the files):

///router var multipart = require("connect-multiparty"); var multipartMiddleware = multipart(); //..... router.post("/save", multipartMiddleware, function(req, res, next) { var s = new Settings(res,req); s.saveSection(); }); //controller var File = require('fileuploader'); //more action this.saveSection = function(){ //more action var files = new File(req.files); log.info(req.files); files.moveUploadedFiles(FOLDER,function(upload){ //moreactions... }); }; //uploader //moreactions.... this.moveUploadedFiles = function(dest,cb){ async.waterfall ([ function (cb) { $this.createFolder(dest,function(){ cb(null); }); } ], function () { var infile = []; var outfile = []; var uploadSize = []; for (var i in files) { (function(i) { var path = dest + '/' + files[i].originalFilename; infile[i] = fs.createReadStream(files[i].path); outfile[i] = fs.createWriteStream(path); infile[i].on('data',function(data) { if(!uploadSize[i]){ uploadSize[i] = 0; } uploadSize[i] += data.length/1024; console.log(files[i].originalFilename + ': '+parseInt(uploadSize[i])+'/'+(parseInt(files[i].size/1024)) + ' Kb'); }); infile[i].pipe(outfile[i]); })(i); } return cb(1); }); };

The bottom line is that the files are first loaded, and then the answer comes and I see in the console the progress of copying them from the temporary directory to the target one. How to make stream at the input somewhere in the controller or at the router level, I don’t know which is better, but in my opinion it’s better to do it in the uploader script. Those. the task is that at the moment of submission of the form, the download starts immediately with the help of stream, so that this progress can be output, but not like now - I can see the progress of copying the file already when it is on the server ... How to implement it? Maybe within the same request, the transferred files have the ability to read asynchronously? or is it not possible?

    2 answers 2

    I believe that you are using async.waterfall . In general, it is not clear what exactly your problem is. But if you need the files to be copied asynchronously, and after all the files have been copied, some function is called, try rewriting this.moveUploadedFiles (I don't know what files , and if this is an associative array, not Array , just make an Array of it beforehand):

     this.moveUploadedFiles = function(dest, callback){ async.waterfall ([ function (cb) { $this.createFolder(dest,function(){ cb(null); }); }, function (cb) { async.each(files, function(file, next) { var path = dest + '/' + file.originalFilename; var totalSize = file.size / 1024; var ws = fs.createWriteStream(path); ws.on('error', function(err) { next('Ошибка при записи файла ' + path + ' (' + err + ')'); }); var uploadSize = 0; var rs = fs.createReadStream(file.path); rs.on('data', function(data) { uploadSize += data.length; console.log(file.originalFilename + ': ' + (uploadSize / 1024) + '/' + totalSize + ' Kb'); }); rs.on('error', function() { next('Ошибка при чтении файла ' + path + ' (' + err + ')'); }); rs.on('end', function() { console.log('A file', path, 'processed'); next(null, file); }); rs.pipe(ws); }, function(err) { if(err) console.log('Failed to process', err); else console.log('All files have been processed successfully'); cb(); }); } ], function (err) { callback(); }); };

    • files is an object containing data about files (req.files from multiparty), async has nothing to do with it, the essence of the problem is nowhere more detailed, stated in the comment to the post @Pavel Mayorov above - sanu0074
    • I understand that async not the main problem. But still you use it incorrectly. But what is your problem is still not clear. Do you want to see the file upload process on the server? Like what it looks like in a client? - Alex Popov
    • Yes, the process of uploading a file to the server. And in my example, the process of copying a file from a temporary folder to the target folder is shown - sanu0074
    • Then socket.io should help you - Alex Popov
    • It is clear that socket.io is needed here, but before that you need at least to output the process of uploading a file to the server at least in the console! - sanu0074

    It looks like multiparty doesn't know how to postpone reading a file. And, in general, it is clear why this happens: the browser can always put the rest of the form data after the file - and then you will not be able to determine in any way where to save the file without first reading it.

    The solution could be a rejection of multiparty and standard browser forms. You can intercept the form submission event on the client, and send a request to the server in an alternative way - put part of the parameters into the request string or into HTTP headers. Then there will be nothing left in the request body except the file itself, and it can be read asynchronously from the request flow.

    • No, no, you probably did not understand the essence of the problem, or perhaps I incorrectly stated it in the question. The problem is not that I expect the file to be recorded (that is, it comes along with all the data in the form), here you do not need to make one separate request containing only files, for example. When the form data comes to the server, by default the file is placed in a temporary folder, then we already do something with it ... so, the main point in the task is to output to the console the progress of writing the file from the form to a temporary folder using stream - sanu0074