Studying multithreading, I wrote a small code that, in theory, should start threads in turn, wrote such a small script:

import queue import threading exit_flag = False class MyThread(threading.Thread): def __init__(self, thread_id, name, q): threading.Thread.__init__(self) self.thread_id = thread_id self.name = name self.q = q def run(self): print("Starting {}".format(self.name)) process_data(self.name, self.q) print("Exiting {}".format(self.name)) def process_data(working_thread, q): while not exit_flag: queue_lock.acquire() if not work_queue.empty(): data = q.get() queue_lock.release() print("{} processing {}".format(working_thread, data)) thread_list = ["Thread-1", "Thread-2", "Thread-3"] name_list = ["One", "Two", "Three", "Four", "Five"] queue_lock = threading.Lock() work_queue = queue.Queue(10) threads = [] t_id = 1 for t_name in thread_list: thread = MyThread(t_id, t_name, work_queue) thread.start() threads.append(thread) t_id += 1 queue_lock.acquire() for word in name_list: work_queue.put(word) queue_lock.release() while not work_queue.empty(): pass exit_flag = True for t in threads: t.join() print("Main thread end!")

What is most interesting, while working in the debugging mode, everything works fine, and if you just take it to start, then nothing happens, just the message that the thread is running. Do not tell me where I make a mistake?

  • in your own words, what is the desired behavior of your code? What are you trying to achieve? - jfs
  • Well, it seems like I created a queue of threads that should work, but the threads start but no actions .. And if you start in debug mode (using PyCharm) with breakpoints, everything seems to start to output the results of the console to the console .. And if you do not start directly, . - E1mir
  • describe not the broken implementation, but what behavior do you expect that can be seen from the outside (for example, a, b, c, d, d should appear on the screen in a random order, but instead nothing is displayed) - jfs
  • The only thing I noticed by the way, when I debugged in debug mode, if I remove a breakpoint, for example, at the beginning of the process_data() function, then the code block is completely ignored and not executed at all, but if you put everything in ok. And what I want, I want all three threads to perform actions that are in the queue - E1mir

3 answers 3

 import queue, threading, time ResultDt = {} ResultLock = threading.Lock() class MyThread(threading.Thread): def __init__(self, q, name): threading.Thread.__init__(self, name=name, daemon=True) self.q = q def run(self): print("Starting {}".format(self.name)) while True: # циклически получать задачи в потоке try: data = self.q.get(timeout=2) # ожидать задачу в течение 2 сек print("{} processing {}".format(self.name, data)) except queue.Empty: print('timeout выход', self.name) break else: # из очереди получена задача try: if data is None: print('None выход', self.name) break else: r = worker(data) # выполнение задачи ResultLock.acquire() # заблокировать ResultDt['run_{}'.format(data)] = r # сохр. результат выполнения ResultLock.release() # освободить finally: # task_done, при любом исходе self.q.task_done() print("Exiting {}".format(self.name)) def worker(arg): """выполнение задач""" time.sleep(.5) return 'done_{}'.format(arg) if __name__ == '__main__': work_queue = queue.Queue() # очередь задач threads = [MyThread(work_queue, t_name) for t_name in ("Thread-1", "Thread-2", "Thread-3")] # потоки for thread in threads: thread.start() for word in ["One", None, "Two", "Three", "Four", "Five"]: # задачи work_queue.put(word) work_queue.join() # заблокировать до выполнения всех задач time.sleep(3) # ожидать более timeout print(ResultDt) print("Main thread end!")

out

 Starting Thread-1 Starting Thread-2 Starting Thread-3 Thread-1 processing One Thread-2 processing None None выход Thread-2 Thread-3 processing Two Exiting Thread-2 Thread-1 processing Three Thread-3 processing Four Thread-1 processing Five timeout выход Thread-3 Exiting Thread-3 timeout выход Thread-1 Exiting Thread-1 {'run_One': 'done_One', 'run_Two': 'done_Two', 'run_Three': 'done_Three', 'run_Four': 'done_Four', 'run_Five': 'done_Five'} Main thread end!
  • Aah, that's how it should have been, thanks) - E1mir

Judging by the accepted answers, you want to run 3 threads and process 5 words in them:

 #!/usr/bin/env python import time from multiprocessing.pool import ThreadPool def handle_word(word): time.sleep(5) return 'run_' + word, 'done_' + word result = dict(ThreadPool(3).map(handle_word, ["One", "Two", "Three", "Four", "Five"]))

A thread pool independently uses a queue inside, providing a simple interface.

  • Wow, I did not know about this)) Thank you !! - E1mir

Instead of threading.Lock() locks, it is better to use Queue features:

Queue.task_done ()

Indicate that a formerly enqueued task is complete. Used by consumer threads. For each get () used to fetch a task, the aforementioned call to task_done () tells you what it is.

If a join () is currently a blocking, it will be taken for every item that has been processed ().

If there were any items placed in the queue.

Documentation: https://docs.python.org/3.6/library/queue.html

There are also examples

  • Good hint, read how it works, thanks) - E1mir