There is a code:

username = 'ivan' parted_list = [[1],[2],[3],[4],[5],[6],[7],[8],[9],[10]] my_function(username,parted_list): print('yep!') p = multiprocessing.Pool() thread_number = 0 for i in range(10): work = p.map(lambda f: f(username,parted_list[thread_number]),[my_function]) thread_number +=1 p.close() p.join()

What is the problem: the code works, but the threads are started in turn, due to the for loop In order for them to run in parallel, as I understand it, this line is necessary:

 work = p.map(lambda f: f(username,parted_list[thread_number]),[my_function])

redo this:

 work = p.map(lambda f: f(username,parted_list[thread_number]),[my_function,my_function,my_function,my_function,my_function,my_function,my_function,my_function,my_function,my_function])

and remove the for loop. BUT! for each function there must be a different thread_number , that without a cycle to get, I do not know how. Is there a solution to this problem? It is also desirable in Python Zen, not to write my_function 10 times, but to do it more concisely.

    1 answer 1

    Use Process and daemon=True Documentation for this.
    Here is an example of your code:

     from multiprocessing import Process from time import sleep def my_function(username, parted_list): print('yep!', username, parted_list) sleep(10) print(parted_list, 'outed') if __name__ == "__main__": tmp = [] thread_number = 0 username = 'ivan' parted_list = [[1], [2], [3], [4], [5], [6], [7], [8], [9], [10]] for i in range(10): p = Process( group=None, target=my_function, args=(username, parted_list[thread_number],), daemon=True ) tmp.append(p) thread_number += 1 for item in tmp: item.start() for item in tmp: item.join()

    Conclusion:

     yep! ivan [1] yep! ivan [2] yep! ivan [4] yep! ivan [5] yep! ivan [10] yep! ivan [6] yep! ivan [7] yep! ivan [9] yep! ivan [3] yep! ivan [8] [2] outed [1] outed [4] outed [5] outed [10] outed [6] outed [7] outed [9] outed [3] outed [8] outed