I want to parallelize the execution of tests on 2 devices.
Those. I have a list with tests, and I want the test_proc function to distribute them across all devices, test1.py to the first device, test2.py to the second, etc., they run in parallel, if one test runs faster, then 3 went to him, etc. under the list with tests.
Suppose we need to execute the parse method in several threads. The following is passed to the input of the method:
def parse(url): ..... We need to process a list of urls links urls
from concurrent.futures import ThreadPoolExecutor def start_parse(urls): with ThreadPoolExecutor(count_thread) as executor: for _ in executor.map(parse, urls): pass Thread libraries / processes have multiple libraries
As for your example, I would give the list of devices as urls. The parse method would be our common methods, and I would turn the list of tests into a queue.
In the general test of the flow, we have one device and we select the test from the test queue until this queue is empty.
For example:
tests = Quque() devs = [devs1, devs2, devs3] def parse(dev): while tests.qsize > 0: test = tests.get() #Получаем тест из очереди. ..... Выполняем данный тест для данного устройства def start_parse(devs): with ThreadPoolExecutor(count_thread) as executor: for _ in executor.map(parse, devs): pass Only in this case, the number of threads = the number of devices, otherwise any of the devices will not receive tests.
You can come up with other implementations.
Source: https://ru.stackoverflow.com/questions/968046/
All Articles