The loop starts a fixed number of processes through multiprocessing.

for (i, data) in enumerate(datas): exit_event = Event() exit_events.append(exit_event) thread = Process(target=func, args=(data,i,exit_event)) thread.start()

Processes perform the same task and they complete their work only with manual interruption. At the very beginning of a code we create Manager

  lock = Lock() manager = Manager() DATA_CONTROL=manager.dict()

And in the func function at the beginning I try to check the existence of the key

  lock.acquire() status = DATA_CONTROL.has_key(data) lock.release()

What I get here is such an error

  Process Process-2: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "script.py", line 311, in func status = DATA_CONTROL.has_key(data) File "<string>", line 2, in has_key File "/usr/lib/python2.7/multiprocessing/managers.py", line 755, in _callmethod self._connect() File "/usr/lib/python2.7/multiprocessing/managers.py", line 742, in _connect conn = self._Client(self._token.address, authkey=self._authkey) File "/usr/lib/python2.7/multiprocessing/connection.py", line 169, in Client c = SocketClient(address) File "/usr/lib/python2.7/multiprocessing/connection.py", line 304, in SocketClient s.connect(address) File "/usr/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 2] No such file or directory

Question - How to avoid mistakes? In principle, you need to implement a variable in which you could write and from which you could read both the main process and new processes.

    1 answer 1

    Give all the code, otherwise it's hard to say where the error is.

    Here is an example in python3. There is a dct dictionary in which data is written by key and several processes that wait for data to appear (each process only checks its own key) and delete them from the dictionary when they appear.

     from multiprocessing import Event, Lock, Process, Manager import time import random def func(dct, lock, event, key): while not event.is_set(): lock.acquire() if key in dct: print("Remove data: {0}".format(key)) del dct[key] lock.release() event = Event() lock = Lock() manager = Manager() dct = manager.dict() procs = [] keys = ['a', 'b', 'c'] for key in keys: procs.append(Process(target=func, args=(dct, lock, event, key))) for proc in procs: proc.start() for i in range(10): dct[random.choice(keys)] = i time.sleep(1) event.set() for proc in procs: proc.join() print("END")