📜 ⬆️ ⬇️

Multithreading in Node.js: module worker_threads

On January 18, the release of the Node.js platform version 11.7.0 was announced . Among the notable changes in this version we can point out the conclusion from the discharge of the experimental module worker_threads, which appeared in Node.js 10.5.0 . Now to use it, the --experimental-worker flag is not needed. This module, since its inception, has remained fairly stable, so the decision was made, reflected in Node.js 11.7.0.

The author of the material, the translation of which we publish, proposes to discuss the possibilities of the worker_threads module, in particular, he wants to talk about why this module is needed, and about how, for historical reasons, multithreading is implemented in Node.js. Here we will talk about what problems are associated with writing multi-threaded JS applications, about the existing ways to solve them, and about the future of parallel data processing using so-called “worker threads” (sometimes called “worker threads”). or simply "workers".

Life in a single threaded world


JavaScript was designed as a single-threaded programming language that runs in a browser. “Single-threading” means that in the same process (in modern browsers we are talking about separate browser tabs) only one set of instructions can be executed at a time.

This simplifies the development of applications, facilitates the work of programmers. Initially, JavaScript was a language suitable only for adding some interactive features to web pages, for example, something like form validation. Among the tasks for which JS was designed, there was not something particularly complex that needed multithreading.

Ryan Dahl , the creator of Node.js, saw an interesting opportunity in this language restriction. He wanted to implement a server platform based on an asynchronous I / O subsystem. This meant that the programmer did not need to work with threads, which greatly simplifies development for a similar platform. When developing programs designed for parallel code execution, problems may arise that are very difficult to solve. For example, if several threads try to access the same memory area, this can lead to the so-called “process race condition” disrupting the operation of the program. Such errors are difficult to reproduce and correct.

Is the Node.js platform single-threaded?


Are Node.js applications single-threaded. Yes, in a way it is. In fact, Node.js allows you to perform certain actions in parallel, but to do this, the programmer does not need to create threads or synchronize them. The Node.js platform and the operating system perform parallel I / O operations with their own means, and when it comes time to process the data with our JavaScript code, it works in single-threaded mode.

In other words, everything except our JS code works in parallel. In synchronous blocks of JavaScript code, commands are always executed one by one, in the order in which they are presented in the source code:

let flag = false function doSomething() {  flag = true  // Тут идёт ещё какой-то код (он не меняет состояние переменной flag)...  // Мы можем быть уверены в том, что здесь в переменную flag записано значение true.  // Какой-то другой код не может поменять эту переменную,  // так как код здесь выполняется синхронно. }

All this is wonderful - in the event that all that our code is doing is performing asynchronous I / O operations. The program consists of small blocks of synchronous code that quickly operate on data, for example, sent to files and streams. The code of program fragments works so fast that it does not block the execution of the code of its other fragments. Much more time than the execution of the code takes to wait for the results of asynchronous I / O operations. Consider a small example:

 db.findOne('SELECT ... LIMIT 1', function(err, result) { if (err) return console.error(err) console.log(result) }) console.log('Running query') setTimeout(function() { console.log('Hey there') }, 1000)

It is possible that the database query shown here will be executed for about a minute, but the Running query message will get to the console immediately after this request is initiated. In this case, the Hey there message will be displayed a second after the execution of the request, regardless of whether it has completed or not yet. Our Node.js application simply calls the function that triggers the request, and the execution of its other code is not blocked. After the request is completed, the application will be notified of this using the callback function, and immediately it will receive an answer to this request.

CPU intensive tasks


What happens if we need to perform heavy calculations using JavaScript? For example - to process a large set of data stored in memory? This can lead to the fact that the program will contain a fragment of synchronous code, the execution of which takes a long time and blocks the execution of other code. Imagine that these calculations take 10 seconds. If we are talking about a web server that processes a request, it will mean that it will not be able to process other requests for at least 10 seconds. This is a big problem. In fact, calculations that are longer than 100 milliseconds can already cause a similar problem.

JavaScript and the Node.js platform were not originally designed for solving tasks that use processor resources intensively. In the case of JS running in a browser, performing such tasks means “brakes” the user interface. In Node.js, this can limit the ability to request the platform to perform new asynchronous I / O tasks and the ability to respond to events associated with their completion.

Let's return to our previous example. Imagine that, in response to a request to the database, several thousand of some encrypted entries came in, which, in a synchronous JS code, need to be decrypted:

 db.findAll('SELECT ...', function(err, results) { if (err) return console.error(err) // Большой объём результатов и их обработка, требовательная к ресурсам процессора. for (const encrypted of results) {   const plainText = decrypt(encrypted)   console.log(plainText) } })

The results, after they are received, appear in the callback function. After that, until the end of their processing, no other JS code can be executed. Usually, as already mentioned, the load on the system created by such a code is minimal, it rather quickly performs the tasks assigned to it. But in this case, the results of the query, which have a considerable amount, came to the program, and we still need to process them. Something like this can take a few seconds. If we are talking about a server that many users are working with, this will mean that they can continue working only after the completion of a resource-intensive operation.

Why in JavaScript never will be flows?


Given the above, it may seem that to solve heavy computational problems in Node.js you need to add a new module that allows you to create and manage threads. How can you do without something like that? Sadly enough, those who use a mature server platform, such as Node.js, do not have the means to beautifully solve tasks related to processing large amounts of data.

All this is true, but if you add the ability to work with streams in JavaScript, this will lead to a change in the very nature of this language. In JS, you cannot simply add the ability to work with threads, for example, in the form of a new set of classes or functions. To do this, you need to change the language itself. In languages ​​that support multi-threading, such a thing as “synchronization” is widely used. For example, in Java, even some numeric types are not atomic. This means that if you do not use synchronization mechanisms to work with them from different threads, all this may result in, for example, after a pair of threads tries to change the value of the same variable at the same time, several bytes of this variable will be set to one a stream, and a little - another. As a result, such a variable will contain something incompatible with the normal operation of the program.

Primitive solution of the problem: iterations of the event loop


Node.js will not execute the next block of code in the event queue until the previous block completes. This means that to solve our problem, we can break it into parts represented by synchronous code fragments, and then use the setImmediate(callback) construction to plan the execution of these fragments. The code specified by the callback function in this construct will be executed after the tasks of the current iteration (tick) of the event loop are completed. After that, the same design is used to queue the next batch of calculations. This allows not to block the cycle of events and, at the same time, to solve voluminous tasks.

Imagine that we have a large array that needs to be processed, while complex processing is required during the processing of each element of such an array:

 const arr = [/*large array*/] for (const item of arr) { // для обработки каждого элемента массива нужны сложные вычисления } // код, который будет здесь, выполнится только после обработки всего массива.

As already mentioned, if we decide to process the entire array in one go, it will take too much time and will not allow another application code to be executed. Therefore, we divide this big task into parts and use the setImmediate(callback) construction:

 const crypto = require('crypto') const arr = new Array(200).fill('something') function processChunk() { if (arr.length === 0) {   // код, выполняющийся после обработки всего массива } else {   console.log('processing chunk');   // выберем 10 элементов и удалим их из массива   const subarr = arr.splice(0, 10)   for (const item of subarr) {     // произведём сложную обработку каждого из элементов     doHeavyStuff(item)   }   // поставим функцию в очередь   setImmediate(processChunk) } } processChunk() function doHeavyStuff(item) { crypto.createHmac('sha256', 'secret').update(new Array(10000).fill(item).join('.')).digest('hex') } // Этот фрагмент нужен лишь для подтверждения того, что, обрабатывая большой массив, // мы даём возможность выполняться и другому коду. let interval = setInterval(() => { console.log('tick!') if (arr.length === 0) clearInterval(interval) }, 0)

Now we, in one go, do the processing of ten array elements, after which, using setImmediate() , we plan to perform the next batch of calculations. And this means that if in the program you need to execute some other code, it can be executed between operations on processing fragments of an array. For this purpose, here, at the end of the example, there is code in which setInterval() .

As you can see, this code looks much more complicated than its original version. And often, the algorithm can be much more complicated than ours, which means that, when implemented, it will not be easy to break up the calculations into parts and understand where, to achieve the right balance, you need to set the setImmediate() call, which plans the next fragment of the calculations. In addition, the code is now asynchronous, and if our project depends on third-party libraries, then we may not be able to break the process of solving a heavy task into parts.

Background processes


Perhaps the above approach with setImmediate() fine for simple cases, but it's far from ideal. In addition, streams are not used here (for obvious reasons) and we also do not intend to change the language for the sake of it. Is it possible to perform parallel data processing without using streams? Yes, it is possible, and for this we need some kind of mechanism for background data processing. The point is to run a task, transfer data to it, and so that this task, without interfering with the main code, would use all that it needs, spend on work as much time as it needs, and then return the results to main code. We need something like the following code snippet:

 // Запускаем script.js в новом окружении, без использования разделения памяти. const service = createService('script.js') // Отправляем сервису входные данные и создаём механизм получения результатов service.compute(data, function(err, result) { // тут будут результаты обработки данных })

The reality is that background processes can be used in Node.js. The point is that you can create a fork of the process and implement the above described scheme of work using the mechanism for exchanging messages between the child and parent processes. The main process can interact with the child process, sending events to it and receiving them from it. Shared memory with this approach is not used. All data exchanged between processes is “cloned,” that is, when changes are made to a copy of this data by one process, these changes to another process are not visible. This is similar to an HTTP request — when a client sends it to a server, the server only receives a copy of it. If processes do not use shared memory, this means that during their simultaneous operation it is impossible to create a “race condition”, and that we do not need to burden ourselves with work with threads. Looks like our problem is solved.

True, in fact it is not. Yes - we have one of the solutions to the problem of performing intensive calculations, but it is, again, imperfect. Creating a fork of a process is a resource-intensive operation. It takes time to complete. In fact, we are talking about creating a new virtual machine from scratch and about increasing the amount of memory consumed by the program, which is due to the fact that the processes do not use shared memory. Given the above, it is appropriate to ask whether it is possible, after performing a certain task, to reuse the fork of the process. We can give a positive answer to this question, but here we must remember that the fork of the process is going to transfer various resource-intensive tasks that will be performed synchronously in it. Here you can see two problems:


In order to solve these problems, we will need several forks, not just one, but we will have to limit their number, since each of them takes away system resources and it takes time to create each of them. As a result, modeled on systems serving database connections, we need something like a pool of processes ready for use. The process pool management system, upon receipt of new tasks, will use free processes to perform them, and when a process copes with the task, it will be able to assign a new one to it. There is a feeling that such a scheme of work is not easy to implement, and, in fact, the way it is. We will use the worker-farm package to implement this scheme:

 // главное приложение const workerFarm = require('worker-farm') const service = workerFarm(require.resolve('./script')) service('hello', function (err, output) { console.log(output) }) // script.js // Этот код будет выполняться в процессах-форках module.exports = (input, callback) => { callback(null, input + ' ' + world) }

Module worker_threads


So, is our problem solved? Yes, it can be said that it has been solved, but with this approach a lot more memory is required than it would be necessary if we had a multi-threaded solution at our disposal. Threads consume far less resources than processes. That is why the worker_threads module appeared in worker_threads

Worker threads run in an isolated context. They exchange information with the main process using messages. This saves us from the problem of the “race condition” that affects multi-threaded environments. In this case, the threads of the workers exist in the same process in which the main program is located, that is, with this approach, much less memory is used compared to using forks of processes.

In addition, working with workers, you can use shared memory. So, specially for this purpose objects of type SharedArrayBuffer . They should be used only in cases when the program needs to perform complex processing of large amounts of data. They allow saving resources required for serialization and de-serialization of data when organizing data exchange between workers and the main program using messages.

Work with worker threads


If you use the Node.js platform up to version 11.7.0, then in order to enable the ability to work with the worker_threads module, you need to use the --experimental-worker flag when starting --experimental-worker

In addition, it is worth remembering that creating a worker (as well as creating a stream in any language), although it requires much less resources than creating a fork of the process, also creates a certain load on the system. Perhaps in your case, even this load may be too large. In such cases, the documentation recommends creating a pool of workers. If you need it, then, of course, you can create your own implementation of such a mechanism, but perhaps you should look for something suitable in the NPM registry.

Consider an example of working with worker threads. We will have the main file, index.js , in which we will create a worker thread and pass it some data for processing. The corresponding API is based on events, but I'm going to use a promise here that is allowed when the first message comes from the worker:

 // index.js // Если вы пользуетесь Node.js старше версии 11.7.0, воспользуйтесь // для запуска этого кода командой node --experimental-worker index.js const { Worker } = require('worker_threads') function runService(workerData) { return new Promise((resolve, reject) => {   const worker = new Worker('./service.js', { workerData });   worker.on('message', resolve);   worker.on('error', reject);   worker.on('exit', (code) => {     if (code !== 0)       reject(new Error(`Worker stopped with exit code ${code}`));   }) }) } async function run() { const result = await runService('world') console.log(result); } run().catch(err => console.error(err))

As you can see, using the workflow threading mechanism is quite simple. Namely, when creating a worker, you need to pass the path to the file with the code of the worker and the data to the Worker constructor. Remember that this data is cloned, not stored in shared memory. After the start of the worker, we expect messages from him, listening to the message event.

Above, by creating an object of type Worker , we passed to the constructor the name of the file with the worker's code - service.js . Here is the code for this file:

 const { workerData, parentPort } = require('worker_threads') // Тут, асинхронно, не блокируя главный поток, // можно выполнять тяжёлые вычисления. parentPort.postMessage({ hello: workerData })

In the code of the worker we are interested in two things. First, we need the data transmitted by the main application. In our case, they are represented by the variable workerData . Secondly, we need a mechanism to transfer information to the main application. This mechanism is represented by the parentPort object, which has a postMessage() method, using which we pass the data processing results to the main application. That's how it all works.

Here is a very simple example, but using the same mechanisms one can build much more complex structures. For example, from a worker's thread, you can send multiple messages to the main thread that carry information about the state of data processing if our application needs a similar mechanism. More from the worker data processing results can be returned in parts. For example, something like this can be useful in a situation where the worker is busy, for example, processing thousands of images, and you, not waiting for them to process them, want to notify the main application about the completion of each of them.

Details about the worker_threads module can be found here .

Web workers


You may have heard of web workers. They are designed for use in the client environment, this technology has existed for quite some time and enjoys quite good support for modern browsers. The API for working with web workers is different from what the Node.js module worker_threads gives us, it’s all about the differences between the environments in which they work. However, these technologies are able to solve similar problems. For example, web workers can be used in client applications to perform data encryption and decryption, compression and decompression. With their help, you can process images, implement computer vision systems (for example, we are talking about face recognition) and solve other similar tasks in the browser.

Results


Модуль worker_threads — это многообещающее дополнение возможностей Node.js. Средствами этого модуля можно, не блокируя серверные приложения, выполнять ресурсоёмкие вычисления. Потоки воркеров очень похожи на традиционные потоки, но, так как они не пользуются разделяемой памятью, они лишены сопутствующих традиционному многопоточному программированию проблем наподобие «состояния гонок». Что выбрать тем, кому подобные возможности нужны прямо сейчас? Возможно, так как модуль worker_threads ещё совсем недавно носил статус экспериментального, пока для выполнения фоновой обработки данных в Node.js стоит взглянуть на нечто вроде worker-farm , запланировав переход на worker_threads после того, как сообщество Node.js накопит больше опыта в работе с этим модулем.

Dear readers! Как вы организуете выполнение тяжёлых вычислений в Node.js-приложениях?

Source: https://habr.com/ru/post/437984/

More articles: